museum-digital/MDNodaHelpers
32
0
Fork 0
Code Issues 6 Pull Requests Releases Wiki Activity
367 Commits 1 Branch 0 Tags
db0aa0671b2572891045e673ea113247da1e9f79
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
jrenslin db0aa0671b Accept empty extract data from Wikidata 2025-08-10 18:43:33 +02:00
jrenslin aa1d72cb64 Merge branch 'master' of gitea:museum-digital/MDNodaHelpers 2025-07-08 17:39:14 +02:00
jrenslin 1798e29186 Validate JSON returned from Wikidata's sparql endpoint, throw an 
exception if it is invalid
 2025-07-08 17:38:34 +02:00
jrenslin 5a44bf6540 Tests: replace assertTrue(true) with expectNotToPerformAssertions() 2025-06-27 15:12:56 +02:00
jrenslin 119f216907 Merge branch 'master' of gitea:museum-digital/MDNodaHelpers 2025-06-08 17:20:24 +02:00
jrenslin 25668b7b16 Ping and reconnect DB in fulltext sync for actors fulltext index 2025-06-08 17:19:47 +02:00
jrenslin 8a31cf216e Add shortened 100x A to list of blacklisted tags 2025-05-22 16:25:27 +02:00
jrenslin ff474341ed Add iconclass terms BB, CC, DD, to blacklist 2025-05-08 16:18:05 +02:00
jrenslin 1051e10732 Prevent ambigious splitting of [0-9]{4}-[0-9]{2} 2025-05-06 22:32:00 +02:00
jrenslin 057cac0f1b Ensure 1903/1904 cannot be split 2025-05-05 17:05:47 +02:00
jrenslin 0053fbe030 Support splitting times like "1. Hälfte des 19. Jahrhunderts" 2025-04-28 17:00:32 +02:00
jrenslin 7a2856ffad Split times in more cases (300-20 BC, 300-4000 CE) 2025-04-08 15:18:32 +02:00
jrenslin 00638152cf Prevent splitting of non-existing exact dates (e.g. 31.04.XXXX) 
Close #35
 2025-04-08 03:48:04 +02:00
jrenslin dba60dbce6 Fix order of split days and months within a single year BCE 
Close #32
 2025-04-07 18:32:14 +02:00
jrenslin f84fe1bca5 Fix type error / reference to values now not consistently existing 
anymore
 2025-04-06 22:56:36 +02:00
jrenslin 423959ac94 Stop early if autotranslation cannot proceed after validation 2025-04-05 00:11:03 +02:00
jrenslin e8edb4a459 Time splitter: Handle first/second half 
Close #31
 2025-04-05 00:09:39 +02:00
jrenslin 8491b62a83 Validate against time errors in autogenerating translations for times 
Close #30
 2025-04-04 20:03:59 +02:00
jrenslin bb2b1c2c32 Update NodaGroup 2025-03-13 00:30:33 +01:00
jrenslin 5054d3c62f Use more rigurous trimming in NodaConsolidatedNamesForPersinst 2025-03-10 04:18:00 +01:00
jrenslin beba838c0d Correctly handle multibype hyphens in XXXX-XXXX 2025-03-10 04:13:59 +01:00
jrenslin 54dd958073 See before 2025-03-10 04:05:00 +01:00
jrenslin 5b99304b5c Accept an additional type of hyphen / dash in time splitting 2025-03-10 03:59:44 +01:00
jrenslin 5cce98f15b Extend tests 2025-03-10 03:20:46 +01:00
jrenslin 5036c77f32 Extend test for getting actor ID by life dates + name 2025-03-10 02:18:28 +01:00
jrenslin e95415be8f Add test for getting actor ID by name with life dates 2025-03-10 01:48:09 +01:00
jrenslin 5192781494 Use Wikipedia API for getting descriptions from Wikipedia rather than 
parsing HTML in Wikidata fetcher

Thanks @awinkler
 2025-03-09 02:08:26 +01:00
jrenslin d9d9f7fcdc Continue refactoring tests for time splitter to run provider-based 2025-02-24 14:02:42 +01:00
jrenslin dbfa0df17f Begin restructuring NodaTimeSplitterTest to use data providers 2025-02-21 10:32:07 +01:00
jrenslin 3409ec7afe Begin adding autotranslation language CRH / Crimean Tatar 
Some formatting is still unclear. See https://forum.museum-digital.info/d/52-additional-languages-for-translations-crimean-tatar/9
 2025-02-18 17:51:36 +01:00
jrenslin 27ac3f255a Minor typing improvements 2025-02-15 13:36:50 +01:00
jrenslin 9d7d53a858 Disallow fetching from Wikidata disambiguation pages 
Close #23
 2025-02-13 22:37:17 +01:00
jrenslin 28f6db67ff Disable XML error warnings when parsing unclean inputs from Wikidata 2025-02-13 21:48:07 +01:00
jrenslin 2f3bc5f2fa Prefer wikipedia page titles over wikidata labels 
Close #28
 2025-02-13 21:38:13 +01:00
jrenslin 39362f537a Merge branch 'master' of gitea:museum-digital/MDNodaHelpers 2025-02-13 17:19:43 +01:00
jrenslin de0357473a Make constant for test language in NodaWikidataFetcherTest public, allowing reuse 2025-02-13 17:19:06 +01:00
jrenslin ef43270fb2 Map suffixes material and technique to their respective tag relation 
types
 2025-02-13 14:04:38 +01:00
jrenslin 338e09f001 Add kannada to list of languages fetched from wikidata 2025-02-13 13:10:45 +01:00
jrenslin 4cf9eaf4fa Remove superfluous params passed to function 2025-02-13 13:10:30 +01:00
jrenslin 18438251a7 Add functions for getting IDs by any translated entry irrespective of 
the language
 2025-02-12 17:15:19 +01:00
jrenslin 1cf0f9858a Add tests for loading translations in NodaWikidataFetcher 2025-02-12 16:02:04 +01:00
jrenslin 1d50027809 Make function getWikidataEntity public 2025-02-12 15:48:52 +01:00
jrenslin d1cee17ef5 Add Telugu to list of languages to fetch in Wikidata fetcher 
Close #24
 2025-02-12 12:47:02 +01:00
jrenslin baf7905e0b Map gender Q207959 
Q207959 is androgyny, mapping is a preliminary solution
 2025-02-03 09:41:16 +01:00
jrenslin 9bf14d7d91 Add search function for getting entries in NodaIDGetter across vocabs 2025-01-31 23:25:40 +01:00
jrenslin a621534136 Update NodaBlacklistedTerms 2025-01-24 13:45:28 +01:00
jrenslin 51fe9a5e45 Cover more edge cases for splitting time names 2025-01-15 11:49:20 +01:00
jrenslin 9c2eaa2929 Allow splitting 1945-48 2025-01-15 10:35:35 +01:00
jrenslin 546c17031a Make NodaImportLogger more resilient, prevent error in case of duplicate import names 2024-12-12 12:43:11 +01:00
jrenslin bf22f5541d Retrieve "displayed subject" relationship from suffix "<Motiv>", "[Motiv]" 2024-12-03 16:07:41 +01:00
jrenslin e036d7881a Add missing strict typing in function params 2024-12-01 22:11:17 +01:00
jrenslin d8db941485 Disallow tags of name "Nichtmünzliches" (de) 2024-11-24 16:08:14 +01:00
jrenslin b7bb7364d4 Ensure duplicate time names can be parsed in NodaTimeSplitter (e.g. 
1.1.2024-1.1.2024)
 2024-11-20 10:02:10 +01:00
jrenslin 4dcd93b947 Better validate input JSON fetched from Wikipedia 2024-11-12 15:36:32 +01:00
jrenslin c72ad51dda Merge branch 'master' of gitea:museum-digital/MDNodaHelpers 2024-11-11 09:11:35 +01:00
jrenslin d6dea3e280 Remove use of SESSION in NodaWikidataFetcher 2024-11-11 09:11:15 +01:00
jrenslin 6f7ad13c4e Add class NodaTagRelationIdentifier for parsing tag relation types from 
input tag names
 2024-11-09 19:44:09 +01:00
jrenslin 48355a6a36 Identify uncertainty before brackets ("Berlin ? (Germany)" > "Berlin 
(Germany)" + Uncertain)
 2024-11-09 18:42:18 +01:00
jrenslin 7cfe752c94 Handle commas when guessing time certainty 2024-11-09 15:40:27 +01:00
jrenslin 29ca05f552 Properly handle commas at the end of names when guessing certainty 2024-11-09 15:33:49 +01:00
jrenslin eb371d4270 Ensure times can be split despite spaces at random points in given name 2024-10-23 18:02:23 +02:00
jrenslin 16f36c0852 Improve test coverage 2024-10-10 14:32:55 +02:00
jrenslin 669a8a1459 Add tests for lookup functions by vocabulary references 2024-10-10 14:16:52 +02:00
jrenslin a9c506497c Respect diacritics when looking up tag, actor, .. IDs 2024-10-10 09:51:28 +02:00
jrenslin 06f13c1a71 Add functions for loading only norm data links from Wikidata for places 
+ actors
 2024-10-03 16:36:30 +02:00
jrenslin cd49f194f2 Refactor wikidata fetcher 2024-10-03 15:56:31 +02:00
jrenslin 9b63a4d95d Refactor parsing of norm data links from Wikidata into a dedicated 
function
 2024-10-03 15:03:38 +02:00
jrenslin 96ba020514 Add function for getting actors' names including life dates in batch 2024-09-28 22:45:12 +02:00
jrenslin c650e57eda Remove references to zoom factor for places in Wikidata fetcher 2024-09-25 15:42:59 +02:00
jrenslin dea09b17cd Add safeguard against question marks entering NodaSplitTime 2024-08-02 03:41:51 +02:00
jrenslin cc0997f412 Add direct validation function for noda mail checker 2024-07-28 03:47:35 +02:00
jrenslin f18e4c3edc Make class constant public 2024-07-19 00:59:21 +02:00
jrenslin f220a77ad7 Remove linking to wikipedia as a noda repository in wikidata fetcher 
Close #21
 2024-07-19 00:49:04 +02:00
jrenslin 58d3569718 Fix edge case 200 b.c. in NodaSplitTime 2024-07-08 01:27:03 +02:00
jrenslin 27528c9cf7 Ran phpcbf over code 2024-07-08 00:48:50 +02:00
jrenslin 205e77da0e Remove group members before deleting group 2024-05-27 03:13:03 +02:00
jrenslin f36938b8dd Add functions for updating / deleting groups 2024-05-27 02:44:28 +02:00
jrenslin cfa9cee60d Add additional option for logging in nodac 2024-05-23 22:58:42 +02:00
jrenslin 83a557b989 Add additional assertion to NodaTimeSplitterTest for more thorough type safety 2024-05-05 22:54:33 +02:00
jrenslin 7f342ed3c4 Merge branch 'master' of gitea:museum-digital/MDNodaHelpers 2024-05-04 01:19:23 +02:00
jrenslin 7d303e219f Fix broken time splitting for year ranges BC 2024-05-04 01:17:33 +02:00
jrenslin ce480f8b9f Add validator for tag descriptions 2024-05-01 17:45:07 +02:00
jrenslin eb14615917 Return start and end date when attempting to split time spans 2024-04-17 00:02:31 +02:00
jrenslin bd775bec45 Refactor time splitter, support computing of dates for time entries 2024-04-16 23:21:34 +02:00
jrenslin 2cdfa2e948 Merge branch 'master' of gitea:museum-digital/MDNodaHelpers 2024-03-24 02:17:22 +01:00
jrenslin 81a7d64e27 Handle Ukrainian year names (2022 p > 2022) 2024-03-24 00:59:30 +01:00
jrenslin 6af51323e7 Categorize gender Q2449503 2024-03-09 22:49:38 +01:00
jrenslin 3c43a3f2d3 Return an integer when checking time ID by stored rewrite 2024-01-30 00:25:50 +01:00
jrenslin 09518a0a6e Add function for getting time ID by stored rewrite 2024-01-30 00:07:31 +01:00
jrenslin 93f8f13e62 Blacklist tag aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 2024-01-15 01:46:21 +01:00
jrenslin 8a8f55b38c Extend NodaGroup to allow adding new ones and linking vocabulary entries 
to one
 2023-12-12 04:29:23 +01:00
jrenslin f3831965a3 Extend NodaTimeSplitter 2023-12-09 23:54:45 +01:00
jrenslin 4a49c7a4e7 Fix erroneous splitting of Hungarian time names with additions 2023-12-09 11:56:15 +01:00
jrenslin 3e9f675fdc Add Slovak to list of languages to load 2023-12-08 16:42:24 +01:00
jrenslin 2ab0e75111 Fix erroneous splitting of "YYYY MMD" in Hungarian 2023-12-08 12:26:59 +01:00
jrenslin 27e259072c Add new class NodaGroup 2023-12-07 23:30:24 +01:00
jrenslin 50cb33720b Allow updating and getting names of vocabulary groups 2023-12-07 19:07:11 +01:00
jrenslin a677c605c5 Use MDMysqli's default blend modes for manticore index setup 2023-12-05 22:10:18 +01:00
jrenslin 681002d844 Extend lists for Ukrainian place hierarchy indicators 2023-12-04 19:22:50 +01:00
jrenslin 40cf5a5112 Set expansion limit, blend_mode in manticore tables 2023-12-04 18:45:24 +01:00