--- title: "Using the WorldCat Search API" # title: "using-the-worldcat-search-api" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Using the WorldCat Search API} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) ``` ```{r setup} # let's load this package before getting started library(libbib) ``` _Intermediate knowledge of R is required to follow the examples here._ _Basic knowledge of how to manipulate `data.table`s if helpful but_ _following the examples should be possible without it._ ## Introduction `libbib`'s ability to communicate to the WorldCat Search API is probably the most helpful capability of the package. As such, an entire vignette dedicated to this powerful tool is warranted. Also of note is that documentation on how to use this API is scattered throughout the web, in different places, and sometimes even no longer available and only accessible through the internet archive. Because of this, the goal of this vignette is not only to document `libbib`'s `worldcat_api_search` function, but to provide a single location for compiling this archived documentation and providing information/examples of what the WorldCat Search API is capable, in general. The terminology/nomenclature of the different WorldCat API offerings is fuzzy and inconsistent. The specific kind of Search API that this package offers usage of can, more specifically, be described as the 'WorldCat SRU Search API version 1'. There is a version 2 of this API, but version 1 hasn't been sunsetted as of yet. First, let's speak of what this API is and what it is not. ## What the WorldCat SRU Search API is This API is allows a developer to search for bibliographic records that are cataloged in WorldCat. The queries are made using [the SRU standard search protocol](https://en.wikipedia.org/wiki/Search/Retrieve_via_URL) (Search/Retrieve via URL) using a standard query syntax called [CQL](https://en.wikipedia.org/wiki/Contextual_Query_Language) (Contextual Query Language). The WorldCat Search API doesn't implement all features of CQL; this vignette will illustrate SRU/CQL only insofar as it is supported by this API. This API is open to libraries that maintain both WorldCat Discovery and OCLC Cataloging subscriptions and needs an API key (called a `WSKey`) to work. A request can be made for a key (if your institution doesn't already have one) via [this link.](https://help.oclc.org/Discovery_and_Reference/WorldCat_Discovery/Troubleshooting/How_do_I_request_a_WSKey_for_the_WorldCat_Search_API) ## What the WorldCat SRU Search API is not This API is not the OpenSearch/Basic API, which doesn't allow field-specific searches and only supports keyword-anywhere searches. This is also not the same as using the [advanced search option on worldcat.org](https://www.worldcat.org/advancedsearch). That option only provides a subset of the bibliographic records in WorldCat and is a far less powerful tool. This API is most akin to using the "expert search" option in OCLC's FirstSearch, using the WorldCat database, but differs in that (a) searching the API is programmatically automate-able, and (b) the API allows for, still, more powerful search queries. ## What `libbib`'s `worldcat_api_search` function provides At its most basic, this function takes a SRU query and returns a `data.table` with most of the bibliographic metadata from the MARCXml that the API returns. We'll see how the behavior of this function can be controlled by specifying certain function parameters. This function also offers assistance with the SRU query syntax. Mainly, the function allows you to substitute the arcane search index codes for more human-readable equivalents, prefixed be a (US) dollar-sign. Examples of these aids will be explained later in the vignette. ## An example of usage and what the function returns Let's look at an example of a simple query and what the returned `data.table` looks like. Specifically, we'll search for "Madame Bovary" by "Gustave Flaubert" and we'll only show the first three results ```{r, echo=FALSE} # Sys.sleep(1) ``` ```{r, eval=FALSE} library(libbib) # load this package result <- worldcat_api_search('$title="Madame Bovary" and $author="Gustave Flaubert"') # get the column names names(result) #> [1] "total_wc_results" "result_number" "oclc" "isbn" #> [5] "issn" "title" "author" "pub_date" #> [9] "lang_code" "bib_level" "record_type" "pub_place_code" #> [13] "publisher" "leader" "oh08" "query" # show the first three results result[1:3,] #> total_wc_results result_number oclc isbn issn #> #> 1: 986 1 1125170419 9782253183464 #> 2: 986 2 1049849403 9788415618843 #> 3: 986 3 1203070641 9781664921993 #> title author pub_date lang_code bib_level #> #> 1: Madame Bovary : Flaubert, Gustave, 2019 fre Monograph/Item #> 2: Madame Bovary / Flaubert, Gustave, 2019 spa Monograph/Item #> 3: Madame Bovary / Flaubert, Gustave, 2021 eng Monograph/Item #> record_type pub_place_code publisher #> #> 1: Language Material fr le Livre de poche, #> 2: Language Material sp #> 3: Nonmusical sound recording ohu #> leader oh08 #> #> 1: 00000cam a2200000Mi 4500 190619s2019 fr a g 000 1 fre d #> 2: 00000cam a2200000Ii 4500 180827t20192018sp a 000 1 spa d #> 3: 00000cim a2200000Mi 4500 201104s2021 ohunnnneq f n eng d #> query #> #> 1: srw.ti="Madame Bovary" and srw.au="Gustave Flauber... #> 2: srw.ti="Madame Bovary" and srw.au="Gustave Flauber... #> 3: srw.ti="Madame Bovary" and srw.au="Gustave Flauber... ``` This should give you an idea of the rich information returned by the results `data.table`. All of the information returned is from (or derived from) the MARCXml that the API returns, save for - `total_wc_results` Which returns the number of the results the query yields in WorldCat even if the number of records/results requested is lower than this number. - `result_number` This number of the result, which is helpful if you're using a starting position other than `1` (the default) - `query` The final SRU query (after any of the SRU query assistance routines step in) that is sent to the API. This can be useful for debugging. Hereafter, the output of the example queries will be abbreviated, truncated, elided, or only show a subset of columns in order to save space and aid in following the guide. ## What does an error look like If you craft a query that yields no results, a message telling you such will be displayed, and the return value is `NULL`. If you make an error in the query syntax, no results will be returned, but a diagnostic message returned from the server may tell you what went wrong. Here are two examples... ```{r, echo=FALSE} # Sys.sleep(1) ``` ```{r, eval=FALSE} # missing ending double quotes worldcat_api_search('$title="Madame Bovary and $langauge=greek') #> Received diagnostic message: Query syntax error (org.z3950.zing.cql.CQLParseException: expected index or term, got EOF) #> no results found #> NULL # "$titley" is not a valid search index worldcat_api_search('$titley="Madame Bovary" and $langauge=greek') #> Received diagnostic message: Unsupported index (srw.tiy) #> no results found #> NULL worldcat_api_search("$holding_library=NYP") #> Received diagnostic message: Limit index, can only be used to narrow a result #> for a non-limit index (srw.li) #> no results found #> NULL ``` This last one failed because `$holding_library` is considered a _limit index_. This means that you can't use it to search directly, you can only combine with search facet with other non-limiting indexes to filter the results. Hopefully this will make more sense as you go through this vignette. ## `worldcat_api_search` by example / quick start In guides like this, there is frequently tension between wanting to be a complete reference, and wanting to show cool/helpful examples at a glance. In service of easing this tension, we'll first look at some examples illustrating things this function can do and _then_ turn our attention to an adequate explanation of boolean operators, relations operators, search indexes, etc... These examples are largely taken from the `examples` section of the R documentation for this function. These examples (and the ones further along in this vignette) will also use `data.table` syntax to limit number of rows and columns returned to aid reading. We'll be focusing on the types of queries you can use in this section; controlling the behavior of the function via changing parameters will come later. ```{r, eval=FALSE} library(libbib) # title search for "The Brothers Karamazov" results <- worldcat_api_search('$title="Brothers Karamazov"') # Madame Bovary by Gustave Flaubert in Greek sru <- '$author="Gustave Flaubert" and $title="Madame Bovary" and $language=greek' results <- worldcat_api_search(sru) # Hip Hop materials on wax, cassette, or CD published from # 1987 to 1990 sru <- '(($material_type=cas or $material_type=cda or $material_type=lps) and $subject="Rap") and $year="1987-1990"' results <- worldcat_api_search(sru) # keyword search for "Common Lisp" for materials held # at The New York Public Library sru <- '$keyword="common lisp" and $holding_library=NYP' results <- worldcat_api_search(sru) # keyword search for "Common Lisp" for materials held # by any of the members of the "Manhattan Research # Library Initiative" (MaRLI) joint borrowing program # (New York Public Library, Columbia University, and # New York University) sru <- '($keyword="common lisp" and $holding_library=NYP) or ($keyword="common lisp" and $holding_library=ZYU) or ($keyword="common lisp" and $holding_library=ZCU)' results <- worldcat_api_search(sru) # Books (only books) about Ethics (by dewey division 170s or # LC call number subject class "BJ") published in the 19th # century sru <- '($dewey="17*" or $lc_call="bj*") and $year="18*" and $material_type=bks' results <- worldcat_api_search(sru) # Materials on Musicology (by Dewey division 780s) at # the New York Public Library and not held by any # other insitution sru <- '$dewey="78*" and $holding_library=NYP and $library_holdings_group=11' results <- worldcat_api_search(sru) # Search for materials on "Danger Music" published since 2010 results <- worldcat_api_search('$keyword="danger music" and $year="2010-"') ``` Now that we've seen these examples, sans explanation, we can now have a closer look into the components of a query. Broadly speaking, there are four concepts to be aware of... - relations operators - boolean operators - wildcards - search indexes We'll be looking at each in this order, because I think that makes the most sense. To illustrate the first three concepts, though, we have to use search indexes, before a formal explanation of what they are. Briefly, a search index is a facet along which to search. In all of the examples above, the search indexes were prefixed by a `$` character (e.g. `$title`, `author`, `$keyword`, etc...) ## Relations operators There are four relations operators available for use... - `=` - `exact` - `any` - `all` It should be noted that not every relations operator is available for use with every search index. ### `=` This was the most common operator used in the quick-start examples above. Though this operator can be most fully understood via contrast with the next operator, `exact`, suffice it to say, for now, that using `=` means that all of your search terms must match, without intervening words. This is sometimes referred to as an "un-anchored" search. If the phrase you're search for has spaces in it, you have to surround it with double quotes. Since the SRU query to the `worldcat_api_search` function must be a string, and strings can be made with single quotes and double quotes, we need to surround the entire query with single quotes. We can get around this by "escaping" the quotes, but using only single quotes to surround the whole query is the most elegant approach. You can use both spaces and `+` to separate the search index and the search term on both sides of the relations operator... ```{r, echo=FALSE} # Sys.sleep(1) ``` ```{r, eval=FALSE} sru <- '$title+=+"Common Lisp"' results <- worldcat_api_search(sru) results[1:3, .(total_wc_results, title)] #> total_wc_results title #> #> 1: 367 Common Lisp / #> 2: 367 Practical Common Lisp / #> 3: 367 Common LISP / ``` is the same as... ```{r, echo=FALSE} # Sys.sleep(1) ``` ```{r, eval=FALSE} sru <- '$title = "Common Lisp"' results <- worldcat_api_search(sru) results[1:3, .(total_wc_results, title)] #> total_wc_results title #> #> 1: 367 Common Lisp / #> 2: 367 Practical Common Lisp / #> 3: 367 Common LISP / ``` But we'll be using spaces here. The `=` relations operator is the only one that doesn't require any space or `+` on either side of the operator, and we'll be using both the spaced version and non-spaced version in these examples for this operator, only. ### `exact` (anchored search) The `exact` operator, in contrast with the `=` operator, signals to the API that your search must match as an exact phrase, without any other terms in the matching string. For this reason, this operator is sometimes referred to as an "anchored search". Here's an example using the same title search as above... ```{r, echo=FALSE} # Sys.sleep(1) ``` ```{r, eval=FALSE} results <- worldcat_api_search('$title exact "Common Lisp"') results[1:3, .(total_wc_results, title)] #> total_wc_results title #> #> 1: 37 Common Lisp / #> 2: 37 Common LISP / #> 3: 37 Common LISP / ``` Note that the title "Practical Common Lisp" was returned by the `=` operator query. The `exact` operator will not match this title since it has another word/phrase in it besides "Common Lisp". Using an `exact` search is probably a better fit than `=` for some indexes and in some situations. For example, if you know the exact title of a book (e.g. `$title exact "Brothers Karamazov")`, using `exact` will ensure that books with a title like _"A Guide to The_ _Brothers Karamazov"_ or _"The Brothers Karamazov in Pictures"_ are not returned by the search. That being said, we'll be using the `=` most heavily, and it's easier to read. ### any Using `any` means that any of your search terms (inside the double quotes) can match. For example... ```{r, echo=FALSE} # Sys.sleep(1) ``` ```{r, eval=FALSE} results <- worldcat_api_search('$title any "Common Lisp"') results[1:3, .(total_wc_results, title)] #> total_wc_results title #> #> 1: 351777 An Inquiry into the Human Mind on the Principles o... #> 2: 351777 The book of common prayer. #> 3: 351777 Dictionary of Phrase and Fable ``` matches the title "The book of common prayer" (decidedly _not_ a book of the subject of Common Lisp) since it has the word "common" in it. (Note that capitalization doesn't matter). Using `any` in this context is tantamount to using an `or` boolean operator, which we'll look at in the next section... ```{r, echo=FALSE} # Sys.sleep(1) ``` ```{r, eval=FALSE} results <- worldcat_api_search('$title = Common or $title = "Lisp"') results[1:3, .(total_wc_results, title)] #> total_wc_results title #> #> 1: 351777 An Inquiry into the Human Mind on the Principles o... #> 2: 351777 The book of common prayer. #> 3: 351777 Dictionary of Phrase and Fable ``` (Note the same number and order of results) ### `all` Using the `all` operator means that _all_ of the search terms must match but the search terms can be in any order and have intervening terms in between. For example, search like `$title all "Common Lisp"` can match a (fictional) book named _"Speaking with a lisp is common"_. ### A quick note before continuing If your search term is one word, you do not have to use double quotes to surround the term. For example, `$title = Ethics` and `$title exact Ethics` work perfectly well. If your search term as a single quote in it, it must be escaped so that R doesn't interpret it as the end of the search string. That being said, it appears as if you can just drop the single quote and the search will carry on perfectly fine... ```{r, echo=FALSE} # Sys.sleep(1) ``` ```{r, eval=FALSE} results <- worldcat_api_search('$title exact "Finnegans Wake"') results[1:3, .(total_wc_results, title, query)] #> total_wc_results title query #> #> 1: 761 Finnegans wake / srw.ti exact "Finnegans Wake" #> 2: 761 Finnegans Wake srw.ti exact "Finnegans Wake" #> 3: 761 Finnegans wake srw.ti exact "Finnegans Wake" # yields the same results as results <- worldcat_api_search('$title exact "Finnegan\'s Wake"') results[1:3, .(total_wc_results, title, query)] #> total_wc_results title query #> #> 1: 761 Finnegans wake / srw.ti exact "Finnegan's Wake" #> 2: 761 Finnegans Wake srw.ti exact "Finnegan's Wake" #> 3: 761 Finnegans wake srw.ti exact "Finnegan's Wake" ``` I suspect single quotes are automatically elided by the API. ## Boolean operators (and operator precedence) As evinced in the examples shown earlier in this vignette, you can use the boolean operators __and__, __or__, and __not__ to refine your search. As you mix different boolean operators in a single query, care must be taken to ensure that the order/precedence of these operators matches your intention. For example, in one of the examples above, we searched for Hip Hop materials on wax, cassette, or CD published from 1987 to 1990 with the following SRU query ``` '(($material_type=cas or $material_type=cda or $material_type=lps) and $subject="Rap") and $year="1987-1990"' ``` It's important to note that if we wrote the query like shown below ``` $material_type=cas or $material_type=cda or $material_type=lps and $subject="Rap" and $year="1987-1990"' ``` without any parentheses, the meaning of the query would be ambiguous. Make sure you use parentheses around distinct sections of your search incantation to disambiguate the query. ## Wildcards _taken verbatim from the archived documentation_ For right truncation use an asterisk - * There must be at least three characters before the * for the query to work. There is no left truncation. To wildcard a single character wildcard use a number sign - #. So a query for wom#n provides results that include both woman and women in the results. For a 0-9 number of characters as wildcard characters use ?n. So a query for colo?1r provides results of color and colour. To wildcard characters within (or at the end) a term use the question mark - ?. So a query for colo?r provides results of color, colour, colonizer, and colorimeter. ## Table of all available search indexes | Formal search index code | libbib alias | |--------------------------|--------------| | srw.kw | $keyword | | srw.ti | $title | | srw.ln | $language | | srw.au | $author | | srw.yr | $year | | srw.su | $subject | | srw.li | $holding_library | | srw.mt | $material_type | | srw.no | $oclc | | srw.lc | $lc_call | | srw.dd | $dewey | | srw.dn | $lccn | | srw.bn | $isbn | | srw.in | $issn | | srw.cg | $library_holdings_group | | srw.la | $language_code | | srw.pl | $place_of_publication | | srw.pb | $publisher | | srw.am | $access_method | | srw.cn | $corporate_conference_name | | srw.pc | $dlc_limit | | srw.dt | $document_type | | srw.gn | $government_document_number | | srw.mn | $music_publisher_number | | srw.nt | $notes | | srw.on | $open_digital_limit | | srw.pn | $personal_name | | srw.se | $series | | srw.sn | $standard_number | ## Selected Search Indexes in focus ### $title / srw.ti According to the archived Search API documentation, (I have strong doubts) the title search phrase will automatically elide certain common and "un-important" words. In the field of Natural Language Processing, we call these "stop words". The archive docs indicates that the following words will be removed from the search phase: a, als, am, an, are, as, at, auf, aus, be, but, by, das, dass, de, der, des, dich, dir, du, er, es, for, from, had, have, he, her, his, how, ihr, ihre, ihres, im, in, is, ist, it, kein, la, le, les, mein, mich, mir, mit, of, on, sein, sie, that, the, this, to, un, une, von, was, wer, which, wie, wird, with, yousie, that, the, this, to, un, une, von, was, wer, which, wie, wird, with, you. This means that title searches for _The Brothers Karamazov_, _La Noche Boca Arriba_, _Das Spiel ist aus_, or _Das Kapital_ will, alledgely, internally use the seach phrases _Brothers Karamazov_, _Noche Boca Arriba_, _Spiel_, and _Kapital_, respectively. When searching the title index, though, I'd leave these words in the SRU search query for two reasons... - It makes the title search phrase more readable, _especially_ with non-english phrases. - I have strong doubts that all of these words are elided. For example, a search for _Das Spiel ist aus_, yields results as if the search phrase were indeed _Das Spiel ist aus_, and not as if it were simply _Spiel_. ### $holding_library / srw.li As shown in one of the examples above, the holding library search term is the official OCLC designator. You can search for an institution's code by the institutions name [using this link.](https://www.oclc.org/en/contacts/libraries.html) ### $material_type / srw.mt An exhaustive crosswalk of all material types and their (normally 3-letter) codes would be too large to include here, but you can access the crosswalk in the documentation that is archived [at this link](https://web.archive.org/web/20210426232556/https://help.oclc.org/Librarian_Toolbox/Searching_WorldCat_Indexes/Bibliographic_records/Material_Type_names_and_codes?sl=en%3E) ### $library_holdings_group / srw.cg The "library holdings group" search term is, perhaps, a confusing one. The table below is a crosswalk between the search codes to use in the SRU query, and what they mean. | Search code | Meaning | |-------------|---------| | 05 | 5 or more holdings | | 06 | 10 or more holdings | | 07 | 50 or more holdings | | 08 | 100 or more holdings | | 09 | 500 or more holdings | | 10 | No holdings | | 11 | 1 holding only | | 12 | 2 – 4 holdings | | 13 | 5 – 9 holdings | | 14 | 10 – 24 holdings | | 15 | 25 – 49 holdings | | 16 | 50 - 74 holdings | | 17 | 75 – 99 holdings | | 18 | 100 - 149 holdings | | 19 | 150 - 199 holdings | | 20 | 200 - 299 holdings | | 21 | 300 - 399 holdings | | 22 | 400 - 499 holdings | | 23 | 500 - 599 holdings | | 24 | 600 - 699 holdings | | 25 | 700 - 799 holdings | | 26 | 800 - 899 holdings | | 27 | 900 - 999 holdings | | 28 | 1,000 - 1,499 holdings | | 29 | 1,500 - 1,999 holdings | | 30 | 2,000 - 2,499 holdings | | 31 | 2,500 or more holdings | So, for example, if you wanted to limit your search to items that are held by between 100 to 149 institutions, you would have to add `$library_holdings_group=18` to your SRU query. ### others For all the available search indexes, it is very helpful to know which exact MARC fields are searched in each one. [This archived documentation link](https://web.archive.org/web/20130915101232/http://oclc.org/developer/documentation/worldcat-search-api/complete-list-indexes) contains that information. On that page, the SRU index codes have a prefix of `sru`, but it's really `srw`. Consult the _Table of all available search indexes_ above to find the libbib aliases for the indexes of interest (though, of course, you can use the un-translated index codes if you'd like, too). ----- ## Options in the `worldcat_api_search` function Besides for, of course, the SRU query, the `worldcat_api_search` function takes a number of optional parameters that can be used to alter its semantics. Below is a list and explanation of each of those parameters. (This information is also available by running `help("worldcat_api_search")` in an R console after loading the `libbib` package.) - __max_records__ The maximum number of search results to return. This must be a number between 0 and 100 or `Inf`. If `Inf`, the function will automatically make all follow-up requests to retrieve all search results. To limit the number of times the API is hit, the default is 10 search results. - __sru_query_assist__ A logical (boolean) indicating whether translation from more human-readable aliases to the SRU search index codes should be allowed. The default is `TRUE`. It can also be set to `FALSE`. You can control this parameter globally by setting `options("libbib.sru_query_assist")`. - __frbrGrouping__ FRBR (_Functional Requirements for Bibliographic Records_) is a conceptual framework for understanding the relationships between a "work" (e.g. a novel as planned by an author), its "expression" (e.g. the manuscript of that novel), it's "manifestation" (e.g. the first published version of the novel), and, finally, an "item" (e.g an actual physical book [or microform; audio file; etc...] of the work). With the `frbrGrouping` parameter set to `on` (default), an attempt is made by the WorldCat API to group together similar editions and present only the top held record as the representative record for that group. This, conceptually, can be viewed as an attempt to return search results for any expression of the "work" (or works) referred to in the search query. - __start_at__ The search result to start at (default is 1) - __wskey__ A WorldCat API key. This function is easiest to use by setting the wskey globally, with the following incantation: `options(libbib.wskey="YOUR WSKEY GOES HERE")` - __more__ A logical indicating whether more information from the MARCXML search results should be returned (publisher, bib level, etc....). The default is `TRUE`. - __print.progress__ A logical indicating whether a message should be displayed for each API request. If `max_records` is `Inf` a message will be displayed for every group of 100 search results the function fetches. The default is `TRUE`. - __debug__ A logical indicating whether the HTTP and API responses should be printed (for debugging). The default is `FALSE`. ## Combining this with the other API search functions in `libbib` There are numerous ways to combine the `worldcat_api_search` with the other functions that `libbib` provides to do some really useful investigations. In the example below, will be using the `worldcat_api_search` and `worldcat_api_locations_by_oclc` functions to get a list of institutions that hold any edition of my textbook. (This example uses some `data.table` specific syntax for brevity, but it will work with base R [or "tidyverse"] translations just fine.) First, let's use the search function to get a list of all search results for the book... ```{r, eval=FALSE} results <- worldcat_api_search('srw.ti="Data Analysis with r" and srw.au=fischetti', max_records=Inf) # inspect some of the columns in the first 5 results results[1:5, .(total_wc_results, result_number, oclc, title, author, pub_date) #> total_wc_results result_number oclc title #> #> 1: 11 1 1005106045 DATA ANALYSIS WITH R -. #> 2: 11 2 1089176194 Data analysis with R : #> 3: 11 3 949229431 Data analysis with R : #> 4: 11 4 1242682069 Data Analysis with R #> 5: 11 5 1242707288 Data Analysis with R #> author pub_date #> #> 1: FISCHETTI, TONY. 2018 #> 2: Fischetti, Tony. 2018 #> 3: Fischetti, Tony. 2015 #> 4: Fischetti, Tony 2015 #> 5: Fischetti, Tony 2015 ``` Now let's get all the unique OCLC numbers from all the search results. ```{r, eval=FALSE} all_the_oclcs <- results[, unique(oclc)] all_the_oclcs #> [1] "1005106045" "1089176194" "949229431" "1242682069" "1242707288" #> [6] "1244405806" "1104264768" "1242685020" "1242707684" "1244406814" #> [11] "1104846312" ``` On to the `worldcat_api_locations_by_oclc` function! Since this function takes one OCLC number at a time, we need to use a looping-construct to run the function with _all_ the OCLC numbers in `all_the_oclcs`. We'll be using the `pblapply` function (from the great `pbapply` package) to do this because we get a useful progress bar with no extra effort. ```{r, eval=FALSE} holds <- pblapply(all_the_oclcs, function(x){ worldcat_api_locations_by_oclc(x, include.bib.info=FALSE) }) ``` Since the `pblapply` function returns a list of `data.tables` (on for each OCLC), we'll use `data.table`'s `rbindlist` function to one `data.table` containing all the results. ```{r, eval=FALSE} all_holdings_dt <- rbindlist(holds) all_holdings_dt[1:3] #> oclc institution_identifier institution_name copies #> #> 1: 1005106045 FEM The Ferguson Library 1 #> 2: 1005106045 YDX YBP Library Services 1 #> 3: 1005106045 DUQ Duquesne University Library 1 all_holdings_dt[, .(institution_name)] #> institution_name #> #> 1: The Ferguson Library #> 2: YBP Library Services #> 3: Duquesne University Library #> 4: Centennial College #> 5: George Brown College #> --- #> 1029: Hochschule Mittweida (FH), Hochschulbibliothek #> 1030: Cyberlibris #> 1031: Cyberlibris #> 1032: Cyberlibris #> 1033: Bibliothèque de l'Université du Québec à Trois-Riv... ``` There you have it! My textbook is held by 1033 distinct OCLC institutions! Although the example above only searches the holding institutions of one specific book, the idiom is most helpful/interesting/cool when used for finding the holding institution of a entire class of materials. For example, in a recent project for a curator at my institution, I used the Search API to find all search results for materials on a very specific topic, got all the holding institutions for each of the search results, and then aggregated the institutions (with this package's `dt_counts_and_percents` function) to find the institutions holding the most items on this particular (very specific) topic. ----- ## Sundry external / archived documentation links - __WorldCat Search API current documentation__ _scroll to SRU_ Current documentation on the (SRU) WorldCat Search API call. Contains all index names and codes, parameters available, and the meaning of API status code return numbers - __WorldCat Search API > Using the API > Request Types > SRU__ _(archived OCLC documentation link from 2013)_ Examples of simple SRU requests and explanation of `frbrGrouping` and `servicelevel` parameters - __WorldCat Search API > Indexes__ _(archived OCLC documentation link from 2013)_ Explation of relations operators, boolean operators, title index stop-words (removed words), the MARC subfields search by (only) the subject index, and explation of the wildcard characters `*`; `#`; and `?`. - __WorldCat Search API > Indexes > Complete List of Indexes__ _(archived OCLC documentation link from 2013)_ Explation of relations operators and a complete list of all indexes, their respective SRU index code, the relations available for use with each, and the MARC fields search for each index. Here, the SRU index codes have a prefix of `sru`, but it's really `srw`. - __Material Types Names and Codes__ A complete list of Material Types you can search for, their codes For use with the `srw.mt`/`$material_type` index. [Material Type names and codes - OCLC Support](https://help.oclc.org/Librarian_Toolbox/Searching_WorldCat_Indexes/Bibliographic_records/Material_Type_names_and_codes?sl=en) [Archived from original for posterity](https://web.archive.org/web/20210425140401/https://help.oclc.org/Librarian_Toolbox/Searching_WorldCat_Indexes/Bibliographic_records/Material_Type_names_and_codes?sl=en>) - __WorldCat Search API > Indexes > Tips for specific indexes__ _(archived OCLC documentation link from 2013)_ Contains a lot of very helpful information about some of the different search indexes, the stop words they use, normalization rules, and a very helpful cross-walk on the "Number of Holding Libraries" index. - __WorldCat Search API > Using the API > Parameters__ _(archived OCLC documentation link from 2013)_ Helpful information about what the different SRU search parameters mean. Remember that the parameters of the `worldcat_api_search` function (a) only contain a subset of these, and (b) may have slightly different names - __Finding Institution Codes__ A search box for searching institution names and returning instituion codes suitable for use in the holding institution search index. - __Searching WorldCat Indexes__ _(archived OCLC documentation link from 2012)_ Information about search indexes that are largely available in the above links. Not related to the (SRU) Search API but could be helpful, anyway. - __CQL specification__ Helpful information about CQL query syntax. Not all is applicable to the WorldCat Search API that is discussed here. - __Searching WorldCat Indexes__ Current page with links containing a lot of information about wielding WorldCat searches. Not all information is application to the WorldCat Search API that is discussed here, and most of the most helpful information is already covered in the links above. ## Similar projects / code - __A Ruby Gem to communicate with the WorldCat Search API__ - __Python scripts using the WorldCat search API__ ## Papers/Articles - __Using the WorldCat API to Develop Data-Driven Decision-Making for Gifts-in-Kind__ - __Destroyer and Preserver, Hear, Oh Hear! Not All Uncirculated Books Must Chariotest to a Dark Wintry Bed: How We Used the Books Must Chariotest to a Dark Wintry Bed: How We Used the OCLC WorldCat Search API to Inform Our Weeding Decisions with OCLC WorldCat Search API to Inform Our Weeding Decisions with Holdings Data Holdings Data__