Commit Graph

74 Commits

Author SHA1 Message Date
1798e29186 Validate JSON returned from Wikidata's sparql endpoint, throw an
exception if it is invalid
2025-07-08 17:38:34 +02:00
5192781494 Use Wikipedia API for getting descriptions from Wikipedia rather than
parsing HTML in Wikidata fetcher

Thanks @awinkler
2025-03-09 02:08:26 +01:00
27ac3f255a Minor typing improvements 2025-02-15 13:36:50 +01:00
9d7d53a858 Disallow fetching from Wikidata disambiguation pages
Close #23
2025-02-13 22:37:17 +01:00
28f6db67ff Disable XML error warnings when parsing unclean inputs from Wikidata 2025-02-13 21:48:07 +01:00
2f3bc5f2fa Prefer wikipedia page titles over wikidata labels
Close #28
2025-02-13 21:38:13 +01:00
de0357473a Make constant for test language in NodaWikidataFetcherTest public, allowing reuse 2025-02-13 17:19:06 +01:00
338e09f001 Add kannada to list of languages fetched from wikidata 2025-02-13 13:10:45 +01:00
1d50027809 Make function getWikidataEntity public 2025-02-12 15:48:52 +01:00
d1cee17ef5 Add Telugu to list of languages to fetch in Wikidata fetcher
Close #24
2025-02-12 12:47:02 +01:00
baf7905e0b Map gender Q207959
Q207959 is androgyny, mapping is a preliminary solution
2025-02-03 09:41:16 +01:00
4dcd93b947 Better validate input JSON fetched from Wikipedia 2024-11-12 15:36:32 +01:00
d6dea3e280 Remove use of SESSION in NodaWikidataFetcher 2024-11-11 09:11:15 +01:00
06f13c1a71 Add functions for loading only norm data links from Wikidata for places
+ actors
2024-10-03 16:36:30 +02:00
cd49f194f2 Refactor wikidata fetcher 2024-10-03 15:56:31 +02:00
9b63a4d95d Refactor parsing of norm data links from Wikidata into a dedicated
function
2024-10-03 15:03:38 +02:00
c650e57eda Remove references to zoom factor for places in Wikidata fetcher 2024-09-25 15:42:59 +02:00
f18e4c3edc Make class constant public 2024-07-19 00:59:21 +02:00
f220a77ad7 Remove linking to wikipedia as a noda repository in wikidata fetcher
Close #21
2024-07-19 00:49:04 +02:00
6af51323e7 Categorize gender Q2449503 2024-03-09 22:49:38 +01:00
3e9f675fdc Add Slovak to list of languages to load 2023-12-08 16:42:24 +01:00
bbbc84015b Fix handling of misassigned lcsh / loc links in NodaWikidataFetcher 2023-10-18 02:46:11 +02:00
9942c58b12 Improve parsing of LOC / LCSH from Wikidata 2023-09-29 16:20:53 +02:00
835da05c38 Use wikidata description as fallback if wikipedia description is not
parsable in Wikidata fetcher

Close #16
2023-09-01 12:43:24 +02:00
12a7937218 Comment out debugging lines in NodaWikidataFetcher 2023-08-31 16:11:37 +02:00
a68a03e628 Improve wikidata fetcher 2023-08-31 16:09:21 +02:00
107a4cd640 Improve NodaWikidataFetcher's loading of descriptions
Close #15
2023-08-31 15:38:12 +02:00
67f7bf9fab Add new functions for linking norm data repositories in batch and use
them in Wikidata fetcher
2023-08-29 17:32:22 +02:00
d0e11c323e Further modularize fetching of translations, add new class
NodaBatchInserter for batch inserting translations
2022-11-18 00:26:23 +01:00
b318b5b471 Better modularize NodaWikidataFetcher's loading of translations 2022-11-14 00:51:56 +01:00
1a9b195067 Fix type safety error 2022-09-15 21:35:36 +02:00
5819caff91 Remove superfluous variable assignments 2022-09-15 21:29:07 +02:00
ece1e44a9e Skip importing uncertain birth years in Wikidata fetcher 2022-07-20 15:50:42 +02:00
ac79f421ff Add Ukranian to list of languages for which to fetch translations 2022-05-12 16:40:37 +02:00
47226b6538 Fix bug caused by missing handling of different retrieval modes for
Wikidata fetcher
2022-04-18 20:45:32 +02:00
d5b593c334 Expect usage of function setRetrivalMode instead of a GET param for
setting retrieval modes in NodaWikidataFetcher
2022-04-18 13:19:00 +02:00
7dde870afb Improve type-safety of wikidata fetcher 2022-04-13 00:16:05 +02:00
6347de2635 Validate Wikidata IDs before attempting to fetch from Wikidata
Close #8
2022-03-05 13:58:18 +01:00
09a5096588 Remove superfluous check for yet undescribed external noda repos 2022-01-18 00:48:32 +01:00
e7f1515227 Use strict comparisons in NodaWikidataFetcher in remaining places 2022-01-16 15:18:04 +01:00
9132745631 Fix bug in time splitter, make code more explicit 2022-01-09 22:19:22 +01:00
109f18e63c Use a more explicit !empty for checking string contents 2022-01-08 14:15:51 +01:00
52a90d669c Validate geonames and TGN IDs fetched from Wikidata 2021-12-14 15:40:07 +01:00
20f609f6d0 Use integers for geonames and TGN IDs 2021-12-14 15:38:44 +01:00
93cd09ed23 Fix bug in preventing impossible noda relations 2021-12-12 03:36:23 +01:00
340bfac96c Prevent attempts to write link noda repositories for the incorrect
linkable types (e.g. iconclass for places)
2021-12-11 15:33:31 +01:00
4a26ab60ca Fix missing URL prefix for iconclass 2021-12-11 15:15:31 +01:00
e00dd08c23 Use ON DUPLICATE KEY update instead of checking value existence with a
separate query
2021-12-11 01:29:48 +01:00
9471a030d5 Remove disabled noda repositories to link 2021-12-11 01:19:57 +01:00
24714265c2 Prevent error if wikidata doesn't return a search result 2021-11-30 17:53:24 +01:00