museum-digital/MDNodaHelpers
32
0
Fork 0
Code Issues 6 Pull Requests Releases Wiki Activity
391 Commits 1 Branch 0 Tags
5b6d7f9401b7e0f77e0f41ad18d108706dd11ad3
Commit Graph
76 Commits
Author SHA1 Message Date
jrenslin 773ae0659c Support importing Wikidata links to the Catalogue of Life 2025-09-23 14:17:33 +02:00
jrenslin db0aa0671b Accept empty extract data from Wikidata 2025-08-10 18:43:33 +02:00
jrenslin 1798e29186 Validate JSON returned from Wikidata's sparql endpoint, throw an 
exception if it is invalid
 2025-07-08 17:38:34 +02:00
jrenslin 5192781494 Use Wikipedia API for getting descriptions from Wikipedia rather than 
parsing HTML in Wikidata fetcher

Thanks @awinkler
 2025-03-09 02:08:26 +01:00
jrenslin 27ac3f255a Minor typing improvements 2025-02-15 13:36:50 +01:00
jrenslin 9d7d53a858 Disallow fetching from Wikidata disambiguation pages 
Close #23
 2025-02-13 22:37:17 +01:00
jrenslin 28f6db67ff Disable XML error warnings when parsing unclean inputs from Wikidata 2025-02-13 21:48:07 +01:00
jrenslin 2f3bc5f2fa Prefer wikipedia page titles over wikidata labels 
Close #28
 2025-02-13 21:38:13 +01:00
jrenslin de0357473a Make constant for test language in NodaWikidataFetcherTest public, allowing reuse 2025-02-13 17:19:06 +01:00
jrenslin 338e09f001 Add kannada to list of languages fetched from wikidata 2025-02-13 13:10:45 +01:00
jrenslin 1d50027809 Make function getWikidataEntity public 2025-02-12 15:48:52 +01:00
jrenslin d1cee17ef5 Add Telugu to list of languages to fetch in Wikidata fetcher 
Close #24
 2025-02-12 12:47:02 +01:00
jrenslin baf7905e0b Map gender Q207959 
Q207959 is androgyny, mapping is a preliminary solution
 2025-02-03 09:41:16 +01:00
jrenslin 4dcd93b947 Better validate input JSON fetched from Wikipedia 2024-11-12 15:36:32 +01:00
jrenslin d6dea3e280 Remove use of SESSION in NodaWikidataFetcher 2024-11-11 09:11:15 +01:00
jrenslin 06f13c1a71 Add functions for loading only norm data links from Wikidata for places 
+ actors
 2024-10-03 16:36:30 +02:00
jrenslin cd49f194f2 Refactor wikidata fetcher 2024-10-03 15:56:31 +02:00
jrenslin 9b63a4d95d Refactor parsing of norm data links from Wikidata into a dedicated 
function
 2024-10-03 15:03:38 +02:00
jrenslin c650e57eda Remove references to zoom factor for places in Wikidata fetcher 2024-09-25 15:42:59 +02:00
jrenslin f18e4c3edc Make class constant public 2024-07-19 00:59:21 +02:00
jrenslin f220a77ad7 Remove linking to wikipedia as a noda repository in wikidata fetcher 
Close #21
 2024-07-19 00:49:04 +02:00
jrenslin 6af51323e7 Categorize gender Q2449503 2024-03-09 22:49:38 +01:00
jrenslin 3e9f675fdc Add Slovak to list of languages to load 2023-12-08 16:42:24 +01:00
jrenslin bbbc84015b Fix handling of misassigned lcsh / loc links in NodaWikidataFetcher 2023-10-18 02:46:11 +02:00
jrenslin 9942c58b12 Improve parsing of LOC / LCSH from Wikidata 2023-09-29 16:20:53 +02:00
jrenslin 835da05c38 Use wikidata description as fallback if wikipedia description is not 
parsable in Wikidata fetcher

Close #16
 2023-09-01 12:43:24 +02:00
jrenslin 12a7937218 Comment out debugging lines in NodaWikidataFetcher 2023-08-31 16:11:37 +02:00
jrenslin a68a03e628 Improve wikidata fetcher 2023-08-31 16:09:21 +02:00
jrenslin 107a4cd640 Improve NodaWikidataFetcher's loading of descriptions 
Close #15
 2023-08-31 15:38:12 +02:00
jrenslin 67f7bf9fab Add new functions for linking norm data repositories in batch and use 
them in Wikidata fetcher
 2023-08-29 17:32:22 +02:00
jrenslin d0e11c323e Further modularize fetching of translations, add new class 
NodaBatchInserter for batch inserting translations
 2022-11-18 00:26:23 +01:00
jrenslin b318b5b471 Better modularize NodaWikidataFetcher's loading of translations 2022-11-14 00:51:56 +01:00
jrenslin 1a9b195067 Fix type safety error 2022-09-15 21:35:36 +02:00
jrenslin 5819caff91 Remove superfluous variable assignments 2022-09-15 21:29:07 +02:00
jrenslin ece1e44a9e Skip importing uncertain birth years in Wikidata fetcher 2022-07-20 15:50:42 +02:00
jrenslin ac79f421ff Add Ukranian to list of languages for which to fetch translations 2022-05-12 16:40:37 +02:00
jrenslin 47226b6538 Fix bug caused by missing handling of different retrieval modes for 
Wikidata fetcher
 2022-04-18 20:45:32 +02:00
jrenslin d5b593c334 Expect usage of function setRetrivalMode instead of a GET param for 
setting retrieval modes in NodaWikidataFetcher
 2022-04-18 13:19:00 +02:00
jrenslin 7dde870afb Improve type-safety of wikidata fetcher 2022-04-13 00:16:05 +02:00
jrenslin 6347de2635 Validate Wikidata IDs before attempting to fetch from Wikidata 
Close #8
 2022-03-05 13:58:18 +01:00
jrenslin 09a5096588 Remove superfluous check for yet undescribed external noda repos 2022-01-18 00:48:32 +01:00
jrenslin e7f1515227 Use strict comparisons in NodaWikidataFetcher in remaining places 2022-01-16 15:18:04 +01:00
jrenslin 9132745631 Fix bug in time splitter, make code more explicit 2022-01-09 22:19:22 +01:00
jrenslin 109f18e63c Use a more explicit !empty for checking string contents 2022-01-08 14:15:51 +01:00
jrenslin 52a90d669c Validate geonames and TGN IDs fetched from Wikidata 2021-12-14 15:40:07 +01:00
jrenslin 20f609f6d0 Use integers for geonames and TGN IDs 2021-12-14 15:38:44 +01:00
jrenslin 93cd09ed23 Fix bug in preventing impossible noda relations 2021-12-12 03:36:23 +01:00
jrenslin 340bfac96c Prevent attempts to write link noda repositories for the incorrect 
linkable types (e.g. iconclass for places)
 2021-12-11 15:33:31 +01:00
jrenslin 4a26ab60ca Fix missing URL prefix for iconclass 2021-12-11 15:15:31 +01:00
jrenslin e00dd08c23 Use ON DUPLICATE KEY update instead of checking value existence with a 
separate query
 2021-12-11 01:29:48 +01:00