10 Commits

Author SHA1 Message Date
0f5341835f Add version check
Close #11
2025-03-05 01:06:44 +01:00
0196476370 Add built-in scheduler when running --webui
Also:
- Default to webui

Close #16
2025-03-04 20:03:34 +01:00
1f81d48aad Open browser when starting server with --webui
Close #15
2025-03-04 14:26:35 +01:00
e3ac9e9b41 Add makefile, add function for getting version 2025-03-04 14:06:07 +01:00
b7923c9091 Finish first version of WebUI
Close #10
2025-03-04 12:05:31 +01:00
dcc46f679b Add option to upload files in webui 2025-03-03 19:42:53 +01:00
a8a6292d2b Begin adding WebUI
See #10
2025-03-02 22:59:51 +01:00
5abc1fe75a Update readme 2025-02-28 03:39:29 +01:00
7cfd3bb1de First working version
- Move from separate dirs for upload to a unified one (identify
  media/metadata files by file extension)
- Prevent uploading when an import is already scheduled
- Allow setting custom, parser-specific settings
- Add CLI
- Implement WebDAV upload
- Implement checking of upload folders for uploadable contents

Close #6, close #7, close #9, close #3, close #1, close #4
2025-02-27 17:29:20 +01:00
9a5c432186 Remove and ignore binary from git 2025-02-26 13:49:04 +01:00
37 changed files with 2436 additions and 117 deletions

3
.gitignore vendored
View File

@@ -1 +1,2 @@
go.work
/go.work
/bin

3
.gitmodules vendored Normal file
View File

@@ -0,0 +1,3 @@
[submodule "src/webui/translation-json"]
path = src/webui/translation-json
url = https://gitea.armuli.eu/museum-digital/translation-json.git

View File

@@ -1,16 +1,35 @@
# Uploader to automate the Upload and Import of Objects in museum-digital via WebDAV
The purpose of this app will be to allow museums to more simply automate the upload of
their collection data to museum-digital. It is thus targetted at museums using museum-digital
their collection data to museum-digital. It is thus targeted at museums using museum-digital
as a publication platform, while using another collection management system.
While collection management systems differ, an automatic import to museum-digital works in
a uniform way. Import data is uploaded via WebDAV, an import is configured using a text file. ([See here](https://de.handbook.museum-digital.info/import/importe-selbst-durchfuehren.html))
a uniform way. Import data is uploaded via WebDAV, an import is configured using a text file.
([See here](https://de.handbook.museum-digital.info/import/importe-selbst-durchfuehren.html))
This part of the automatic importing can thus be solved in a one-fits-all solution here.
Collection management system-specific export scripts are necessary to present the present
app with data to upload. The export itself is thus out of the scope of the tool described here.
## Idea / Very Short Summary / Use Cases
### Automatic Publication from Separate Collection Management System
A museum wants to use museum-digital only for publication, while they use a separate collection
management system. They want to keep their published collections up to date with their local data.
Hence, they automate an export from their local program. To upload, they move the data into a
specified folder. The present application monitors the folder and uploads the data to museum-digital.
### Automating Image Uploads
If a museum produces a lot of digital media files (e.g. lots of scans of books / manuscripts),
it may become tedious to upload them manually. This becomes an issue all the more if there are
lots of uncompressed, high resolution files (again, common with scans).
In this case, it may be useful to use this application to automatically upload the object images
for import using the `ImageByImport` parser - a.k.a. the matching of image files to objects based
on their file name according to pre-defined rules.
## Workflow
### Setup
@@ -26,8 +45,7 @@ It will hence attempt to do so when first run (or unless all relevant informatio
- Auth token for WebDAV access
- Institution ID
- Parser
- Location of metadata files to upload
- Location of images files to upload
- Location of uploadable files
- Decision, whether objects should be imported as public
#### Setup workflow
@@ -63,26 +81,19 @@ It will hence attempt to do so when first run (or unless all relevant informatio
## Commands
- [ ] Upload
- [x] Upload
- [ ] Verify ability to connect
- [ ] Set optional setting (do the settings need to be validated?)
- [ ] List config
- [x] Set optional setting (do the settings need to be validated?)
- [x] List config
- [ ] Remove optional setting (in case an erroneous setting value has been set)
- [ ] `--version` command to check currently installed version
- [ ] Version check (via Gitea release RSS feed)
## Interface
- A command line interface is sufficient for this application
## Libraries/resources to include
- `requests` / HTTP library
- JSON parser
- XML parser (for version/update check)
- WebDAV client: [Python](https://pypi.org/project/webdav4/), [Golang](https://github.com/studio-b12/gowebdav)
Either JSON or XML may also serve to store the config data.
- A command line interface is sufficient for this application. It would be nice to
eventually provide a GUI for running settings as well as to provide some GUI for
warnings to update the app.
## Additional, optional features

10
go.mod
View File

@@ -1,3 +1,13 @@
module gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader
go 1.24.0
require (
github.com/go-co-op/gocron/v2 v2.16.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/jonboulle/clockwork v0.5.0 // indirect
github.com/madflojo/tasks v1.2.1 // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
github.com/rs/xid v1.6.0 // indirect
github.com/studio-b12/gowebdav v0.10.0 // indirect
)

12
go.sum Normal file
View File

@@ -0,0 +1,12 @@
github.com/go-co-op/gocron/v2 v2.16.0 h1:uqUF6WFZ4enRU45pWFNcn1xpDLc+jBOTKhPQI16Z1xs=
github.com/go-co-op/gocron/v2 v2.16.0/go.mod h1:opexeOFy5BplhsKdA7bzY9zeYih8I8/WNJ4arTIFPVc=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/jonboulle/clockwork v0.5.0 h1:Hyh9A8u51kptdkR+cqRpT1EebBwTn1oK9YfGYbdFz6I=
github.com/jonboulle/clockwork v0.5.0/go.mod h1:3mZlmanh0g2NDKO5TWZVJAfofYk64M7XN3SzBPjZF60=
github.com/madflojo/tasks v1.2.1/go.mod h1:/WMv6u3Xb5eyy+aIM76ildaIT166GOxN/jya9oI7dyo=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
github.com/studio-b12/gowebdav v0.10.0 h1:Yewz8FFiadcGEu4hxS/AAJQlHelndqln1bns3hcJIYc=
github.com/studio-b12/gowebdav v0.10.0/go.mod h1:bHA7t77X/QFExdeAnDzK6vKM34kEZAcE1OX4MfiwjkE=

51
main.go
View File

@@ -4,48 +4,39 @@ import (
"slices"
"os"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/cli"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/webui"
)
// Attempts to connect to WebDAV server, verifying that the authentication data
// works and that the server is one for uploads with museum-digital.
// A museum-digital WebDAV directory can be identified by the availability of the
// directories IMPORT_IMG and IMPORT_XML.
// func verifyConnection(config MDWebDavUploaderConfig) bool {
// }
// Generates the contents of the file import_config.txt necessary
// to let museum-digital know about the required import configuration.
func generateImportConfigFile() { // string {
}
// Uploads metadata files to the configured instance of
// museum-digital / musdb.
func uploadMetadataFiles() {
}
// Uploads media files to the configured instance of
// museum-digital / musdb.
func uploadMediaFiles() {
}
// Negotiates commands based on command line options.
func main() {
if slices.Contains(os.Args, "--help") {
} else if slices.Contains(os.Args, "--version") {
cli.ShowVersion();
} else if slices.Contains(os.Args, "--check-update") {
cli.CheckVersionIsUpToDate();
} else if slices.Contains(os.Args, "--show-config") {
cli.ShowCurrentConfig()
} else if slices.Contains(os.Args, "--run-manual-setup") {
cli.RunManualSetup()
} else if slices.Contains(os.Args, "--set-additional-setting") {
// cli.SetAdditionalSetting()
cli.SetAdditionalSetting()
} else if slices.Contains(os.Args, "--upload") {
// upload()
cli.Upload()
} else if slices.Contains(os.Args, "--webdav-remote-list-toplevel") {
cli.ListRemoteToplevel()
} else if slices.Contains(os.Args, "--webdav-remote-list-metadata-dir") {
cli.ListRemoteMetadataDir()
} else if slices.Contains(os.Args, "--webdav-remote-list-media-dir") {
cli.ListRemoteMediaDir()
} else if slices.Contains(os.Args, "--local-list-metadata") {
cli.ListLocalMetadata()
} else if slices.Contains(os.Args, "--local-list-media") {
cli.ListLocalMedia()
} else if slices.Contains(os.Args, "--webui") {
webui.Run()
} else {
cli.RunManualSetup()
webui.Run()
}
}

22
makefile Normal file
View File

@@ -0,0 +1,22 @@
VERSION := $(shell go run main.go --version)
hello:
echo WebDAV uploader for museum-digital
echo Version is $(VERSION)
build:
go build -o bin/museum-digital-webdav-uploader main.go
run:
go run main.go
compile:
echo "Compiling for every OS and Platform"
GOOS=linux GOARCH=amd64 go build -o bin/museum-digital-webdav-uploader-$(VERSION)-linux-amd64 main.go
GOOS=windows GOARCH=amd64 go build -o bin/museum-digital-webdav-uploader-$(VERSION)-windows-amd64.exe main.go
GOOS=darwin GOARCH=amd64 go build -o bin/museum-digital-webdav-uploader-$(VERSION)-macos-amd64 main.go
GOOS=linux GOARCH=arm64 go build -o bin/museum-digital-webdav-uploader-$(VERSION)-linux-arm64 main.go
GOOS=windows GOARCH=arm64 go build -o bin/museum-digital-webdav-uploader-$(VERSION)-windows-arm64.exe main.go
GOOS=darwin GOARCH=arm64 go build -o bin/museum-digital-webdav-uploader-$(VERSION)-macos-arm64 main.go
all: hello build

Binary file not shown.

53
src/cli/Upload.go Normal file
View File

@@ -0,0 +1,53 @@
package cli
import (
"fmt"
"os"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/uploadsrcdir"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/webdavupload"
)
// Integration: Uploads, if there is uploadable data and no import currently
// scheduled.
func Upload() {
config := loadConfigFromFile()
uploadableMetadata, uploadableMedia := uploadsrcdir.GetUploadableFiles(config)
// If there are no files to upload, do nothing
if len(uploadableMetadata) == 0 && len(uploadableMedia) == 0 {
fmt.Println("No uploadable files identified.")
return
}
// Ensure no files are too new too recent - if any are, say so and abort.
if uploadsrcdir.CheckAnyFileIsTooRecent(uploadableMedia) || uploadsrcdir.CheckAnyFileIsTooRecent(uploadableMetadata) {
fmt.Println("Some of the files have been updated too recently (1min). Blocking upload to prevent accidental uploads in the middle of an ongoing digitization effort.")
return
}
// Open WebDAV client
c := webdavupload.GetWebdavClient(config)
// Check that the remote is not currently occupied.
if webdavupload.CheckRemoteIsFree(c) == false {
fmt.Println("The remote is currently occupied (very recent files, or an import config is currently waiting to be processed, are present).")
return
}
// Upload the files
if len(uploadableMetadata) > 0 {
webdavupload.UploadMetadataFiles(c, os.Stdout, uploadableMetadata)
webdavupload.BatchUnlink(os.Stdout, uploadableMetadata)
}
if len(uploadableMedia) > 0 {
webdavupload.UploadMediaFiles(c, os.Stdout, uploadableMedia)
webdavupload.BatchUnlink(os.Stdout, uploadableMedia)
}
// Generate the import config.
fmt.Println("Generating and uploading import configuration")
webdavupload.SetImportConfigToTheRemote(c, config)
fmt.Println("DONE")
}

View File

@@ -3,10 +3,13 @@ package cli
import (
"bufio"
"fmt"
"os"
"strconv"
"strings"
"os"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/configloader"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/uploadsrcdir"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/webdavupload"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/meta"
)
// Interal wrapper around opening a new bufio.Reader and getting an input
@@ -155,10 +158,10 @@ func requestParser() string {
}
// Prompts the user to enter a valid metadata folder.
func requestMetadataFolder() string {
// Prompts the user to enter a valid folder for uploads.
func requestUploadFolder() string {
fmt.Println("\nPlease enter the full filepath of a directory for metadata files / exports from your local collection management system that should be automatically uploaded and imported.")
fmt.Println("\nPlease enter the full filepath of a directory in which the program should look for files to upload.")
fmt.Println("Note that the folder needs to exist.")
input := getConsoleInput()
@@ -169,28 +172,7 @@ func requestMetadataFolder() string {
fmt.Print("----\nThe specific error is:\n")
fmt.Println(validationErr)
fmt.Print("----\n\n")
return requestMetadataFolder()
}
return output
}
// Prompts the user to enter a valid media folder.
func requestMediaFolder() string {
fmt.Println("\nPlease enter the full filepath of a directory for media files / exports from your local collection management system that should be automatically uploaded and imported.")
fmt.Println("Note that the folder needs to exist.")
input := getConsoleInput()
output, validationErr := configloader.ValidateUploadDir(input)
if validationErr != nil {
fmt.Println("This is not a valid metadata folder. Please try again.")
fmt.Print("----\nThe specific error is:\n")
fmt.Println(validationErr)
fmt.Print("----\n\n")
return requestMediaFolder()
return requestUploadFolder()
}
return output
@@ -215,6 +197,20 @@ func requestPublishOnImport() bool {
}
// Wrapper around configloader.StoreConfigToFile that prints the error message
// if storing failed.
func storeConfigToFile(config configloader.MDWebDavUploaderConfig) {
err := configloader.StoreConfigToFile(config, "")
if err != nil {
fmt.Println("\nFailed to store config")
fmt.Println("Reason:")
fmt.Println(err)
panic("")
}
}
// Queries the user for each of the relevant values for setup.
func RunManualSetup() {
@@ -229,42 +225,175 @@ func RunManualSetup() {
config.Mail = requestMail()
config.WebDavAuthToken = requestWebDavAuthToken()
config.Parser = requestParser()
config.MetadataFolder = requestMetadataFolder()
config.MediaFolder = requestMediaFolder()
config.UploadDir = requestUploadFolder()
config.PublishOnImport = requestPublishOnImport()
err := configloader.StoreConfigToFile(config, "")
if err != nil {
fmt.Println("\nFailed to store config")
fmt.Println("Reason:")
storeConfigToFile(config)
fmt.Println("Great! Your configuration has been stored. If you need to set further parser-specific settings, use --set-additional-setting. Else, use --upload.")
}
// Wrapper around configloader.LoadFromFile() that prints errors.
func loadConfigFromFile() configloader.MDWebDavUploaderConfig {
config, requiresSetup, err := configloader.LoadFromFile("")
// Re-run manual setup and then load anew.
if requiresSetup == true {
fmt.Println("Your config is incomplete. Please run the setup command before proceeding.")
RunManualSetup()
config, _, rerunerr := configloader.LoadFromFile("")
if rerunerr != nil {
fmt.Println("Another error occured in loading your configuration:")
fmt.Println(rerunerr)
panic("")
}
return config
} else if err != nil {
fmt.Println("Another error occured in loading your configuration:")
fmt.Println(err)
panic("")
}
fmt.Println("Great! Your configuration has been stored. If you need to set further parser-specific settings, use --set-additional-setting. Else, use --upload.")
return config
}
// Prints the current configuration.
func ShowCurrentConfig() {
config, requiresSetup, err := configloader.LoadFromFile("")
if requiresSetup == true {
fmt.Println("Your config is incomplete. Please run the setup command before proceeding")
return
}
if err != nil {
fmt.Println("Another error occured:")
fmt.Println(err)
panic("")
}
config := loadConfigFromFile()
fmt.Println(config)
}
// Runs the CLI workflow:
// - Check config validity
func SetAdditionalSetting() {
config := loadConfigFromFile()
// Welcome msg
fmt.Println("You are trying to set an additional setting. For this, you will need to enter 1) which setting you want to manipulate and 2) what you want to set it to.")
fmt.Println("-----\n")
// Print parser-specific settings
fmt.Println("Your currently configured parser is: " + config.Parser)
fmt.Println("You can learn more about the available settings for this parser in the parser documentation. Printing it now.")
for _, p := range(configloader.ListParsers()) {
if p.Title == config.Parser {
fmt.Println(p.Comment)
}
}
fmt.Println("\n-----\n")
// Get values
fmt.Println("Now, please enter which setting you would like to manipulate.")
settingKey := getConsoleInput()
fmt.Println("What would you like to set the setting to?")
settingValue := getConsoleInput()
if config.Settings == nil {
config.Settings = make(map[string]string)
}
config.Settings[settingKey] = settingValue
storeConfigToFile(config)
}
// Receives a list of files and an error. If the error exists, that is printed.
// Otherwise, the listed files of a directory are listed.
func printFolderContents(files []os.FileInfo, err error) {
if err != nil {
fmt.Println("Failed to list top level folder contents")
return
}
if len(files) == 0 {
fmt.Print("There are no files to print here.\n")
}
for _, f := range(files) {
fmt.Println(f.Name())
}
}
// Prints the top level folder contents of the WebDAV remote
func ListRemoteToplevel() {
config := loadConfigFromFile()
files, err := webdavupload.ListTopLevelContents(config)
printFolderContents(files, err)
}
// Prints the current contents of the remote metadata directory.
func ListRemoteMetadataDir() {
config := loadConfigFromFile()
files, err := webdavupload.ListMetadataDir(config)
printFolderContents(files, err)
}
// Prints the current contents of the remote media directory.
func ListRemoteMediaDir() {
config := loadConfigFromFile()
files, err := webdavupload.ListMediaDir(config)
printFolderContents(files, err)
}
// Prints the current contents of the local metadata directory.
func ListLocalMetadata() {
fmt.Println("Printing uploadable metadata files")
config := loadConfigFromFile()
files := uploadsrcdir.ListUploadableMetadata(config)
printFolderContents(files, nil)
}
// Prints the current contents of the local media directory.
func ListLocalMedia() {
fmt.Println("Printing uploadable media files")
config := loadConfigFromFile()
files := uploadsrcdir.ListUploadableMedia(config)
printFolderContents(files, nil)
}
// Prints current version.
func ShowVersion() {
fmt.Println(meta.GetVersion())
}
// Prints, if there is a current version.
func CheckVersionIsUpToDate() {
output, err := meta.CheckVersionIsUpToDate()
if err != nil {
fmt.Println("Failed to get RSS stream for most recent entry")
fmt.Print(err)
panic("Aborting")
}
// Empty output has no title
if output.Title == "" {
fmt.Println("Your version is up to date")
} else {
fmt.Println("There is a more recent version, version: " + output.Title)
fmt.Println("\nReleased: " + output.PubDate)
fmt.Println("\nGet it at " + output.Link)
}
}

View File

@@ -28,7 +28,6 @@ func ListParsers() []ParserListItem {
parserListFromApi := []ParserListItem{}
// parserListFromApi := new(ApiParserListFormat)
json.Unmarshal(rawParserList, &parserListFromApi)
print(parserListFromApi)
return parserListFromApi

View File

@@ -14,8 +14,7 @@ type MDWebDavUploaderConfig struct {
WebDavAuthToken string `json:"token"`
InstitutionId int `json:"institution_id"`
Parser string `json:"parser"`
MetadataFolder string `json:"metadata_folder"`
MediaFolder string `json:"media_folder"`
UploadDir string `json:"upload_directory"`
PublishOnImport bool `json:"visible"`
Settings map[string]string `json:"settings"`
}
@@ -125,19 +124,12 @@ func ValidateConfig(conf MDWebDavUploaderConfig) (MDWebDavUploaderConfig, error)
}
conf.Parser = parserLink
// Validate and clean metadata folder
metadataFolder, mFolderErr := ValidateUploadDir(conf.MetadataFolder)
// Validate and clean upload folder
uploadDir, mFolderErr := ValidateUploadDir(conf.UploadDir)
if mFolderErr != nil {
return conf, mFolderErr
}
conf.MetadataFolder = metadataFolder
// Validate and clean media folder
mediaFolder, mediaFolderErr := ValidateUploadDir(conf.MediaFolder)
if mediaFolderErr != nil {
return conf, mediaFolderErr
}
conf.MediaFolder = mediaFolder
conf.UploadDir = uploadDir
return conf, nil

View File

@@ -23,8 +23,7 @@ func getTestConfig() MDWebDavUploaderConfig {
panic("Test failure: Failed to create test dir")
}
input.MetadataFolder = testDir
input.MediaFolder = testDir
input.UploadDir = testDir
return input
@@ -128,13 +127,13 @@ func TestValidateUploaderConfigAcceptsValidParser(t *testing.T) {
}
// Test that ValidateConfig() fails on non-existent folder.
func TestValidateUploaderConfigFailsOnInvalidMetadataFolder(t *testing.T) {
func TestValidateUploaderConfigFailsOnInvalidUploadDir(t *testing.T) {
input := getTestConfig()
input.MetadataFolder = "nonexistentfolder"
input.UploadDir = "nonexistentfolder"
_, err := ValidateConfig(input)
if err == nil {
t.Fatalf("ValidateConfig() does not return an error on an invalid metadata folder")
t.Fatalf("ValidateConfig() does not return an error on an invalid upload folder")
}
}
@@ -145,14 +144,14 @@ func TestWritingAndReadingConfigWorks(t *testing.T) {
input := getTestConfig()
input, _ = ValidateConfig(input)
writeErr := StoreConfigToFile(input, input.MetadataFolder + "/config.json")
writeErr := StoreConfigToFile(input, input.UploadDir + "/config.json")
if writeErr != nil {
t.Log("Error:")
t.Log(writeErr)
t.Fatalf("Failed to write config to override path")
}
loadedFromFile, setupRequired, err := LoadFromFile(input.MetadataFolder + "/config.json")
loadedFromFile, setupRequired, err := LoadFromFile(input.UploadDir + "/config.json")
if setupRequired != false {
t.Fatalf("Expected no setup to be required, but return value indicated thus")
@@ -161,7 +160,7 @@ func TestWritingAndReadingConfigWorks(t *testing.T) {
t.Fatalf("Returned an error on trying to load config file")
}
// Golang can't compare structs with slices or maps
if input.InstanceLink != loadedFromFile.InstanceLink || input.Mail != loadedFromFile.Mail || input.WebDavAuthToken != loadedFromFile.WebDavAuthToken || input.InstitutionId != loadedFromFile.InstitutionId || input.Parser != loadedFromFile.Parser || input.MetadataFolder != loadedFromFile.MetadataFolder || input.MediaFolder != loadedFromFile.MediaFolder || input.PublishOnImport != loadedFromFile.PublishOnImport {
if input.InstanceLink != loadedFromFile.InstanceLink || input.Mail != loadedFromFile.Mail || input.WebDavAuthToken != loadedFromFile.WebDavAuthToken || input.InstitutionId != loadedFromFile.InstitutionId || input.Parser != loadedFromFile.Parser || input.UploadDir != loadedFromFile.UploadDir || input.PublishOnImport != loadedFromFile.PublishOnImport {
t.Log("Input")
t.Log(input)

View File

@@ -0,0 +1,20 @@
package importconfiggen
import (
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/configloader"
)
func GenerateImportConfig(config configloader.MDWebDavUploaderConfig) string {
output := "# Generated by user: " + config.Username
output += "\nmail: " + config.Mail + "\nparser: " + config.Parser
if config.Settings != nil {
for settingKey, settingValue := range(config.Settings) {
output += "\nsetting: " + settingKey + ": " + settingValue
}
}
return output
}

View File

@@ -0,0 +1,47 @@
package meta
import (
"net/http"
"encoding/xml"
)
// Very reduced item, as only title, link and pubDate are useful for a version check.
type Item struct {
Title string `xml:"title"`
Link string `xml:"link"`
PubDate string `xml:"pubDate"`
}
type Channel struct {
Items []Item `xml:"item"`
}
type Rss struct {
Channel Channel `xml:"channel"`
}
// Checks if the current version is up to date or not. If there is a newer / different
// version.
func CheckVersionIsUpToDate() (Item, error) {
resp, err := http.Get("https://gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/releases.rss")
if err != nil {
return Item{}, err
}
defer resp.Body.Close()
rss := Rss{}
decoder := xml.NewDecoder(resp.Body)
decodeErr := decoder.Decode(&rss)
if decodeErr != nil {
return Item{}, err
}
newestItem := rss.Channel.Items[0];
if (newestItem.Title == GetVersion() || newestItem.Title == "v" + GetVersion()) {
return Item{}, nil
}
return newestItem, nil
}

6
src/meta/meta.go Normal file
View File

@@ -0,0 +1,6 @@
package meta
// Returns current version number.
func GetVersion() string {
return "0.1.1"
}

View File

@@ -0,0 +1,130 @@
package uploadsrcdir
import (
"path/filepath"
"fmt"
"io/fs"
"os"
"runtime"
"slices"
"strings"
"sync"
"time"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/configloader"
)
var MetadataFileExts = []string{".xml", ".csv", ".json"}
var MediaFileExts = []string{".jpg", ".jpeg", ".png", ".gif", ".bmp", ".tif", ".tiff", ".pdf", ".mp3", ".mp4"}
// Lists all files in a folder that are of any of the expected file extensions.
func getFilesInDir(startPath string, fileExts []string) []os.FileInfo {
allFiles := []os.FileInfo{}
filepath.WalkDir(startPath, func(path string, entry fs.DirEntry, err error) error {
if err != nil {
return err
}
if slices.Contains(fileExts, filepath.Ext(entry.Name())) || slices.Contains(fileExts, strings.ToLower(filepath.Ext(entry.Name()))) {
// add entry and it's size to dirMap
file, err := entry.Info()
if err != nil {
fmt.Println("Error retrieving file info for:", err)
} else {
allFiles = append(allFiles, file)
}
}
return nil
})
return allFiles
}
// Lists all files to upload in metadata directory
func ListUploadableMetadata(config configloader.MDWebDavUploaderConfig) []os.FileInfo {
return getFilesInDir(config.UploadDir, MetadataFileExts);
}
// Lists all files to upload in metadata directory
func ListUploadableMedia(config configloader.MDWebDavUploaderConfig) []os.FileInfo {
return getFilesInDir(config.UploadDir, MediaFileExts);
}
// Returns lists of uploadable folder contents grouped into a) metadata files and b) media files.
func GetUploadableFiles(config configloader.MDWebDavUploaderConfig) ([]string, []string) {
metadataFiles := []string{}
mediaFiles := []string{}
filepath.WalkDir(config.UploadDir, func(path string, entry fs.DirEntry, err error) error {
if err != nil {
return err
}
if slices.Contains(MetadataFileExts, filepath.Ext(entry.Name())) || slices.Contains(MetadataFileExts, strings.ToLower(filepath.Ext(entry.Name()))) {
metadataFiles = append(metadataFiles, path)
}
if slices.Contains(MediaFileExts, filepath.Ext(entry.Name())) || slices.Contains(MediaFileExts, strings.ToLower(filepath.Ext(entry.Name()))) {
mediaFiles = append(mediaFiles, path)
}
return nil
})
return metadataFiles, mediaFiles
}
// Checks if a list of files have been modified in the last minute.
// If so, returns true.
func CheckAnyFileIsTooRecent(files []string) bool {
now := time.Now()
maxConcTasks := min(10, runtime.NumCPU())
// Set a semaphore to restrict the number of concurrent upload tasks.
semaphore := make(chan struct{}, maxConcTasks)
wg := &sync.WaitGroup{}
output := false
for _, f := range(files) {
semaphore <- struct{}{} // acquire
wg.Add(1)
go func() {
defer wg.Done()
file, fStatErr := os.Stat(f)
if fStatErr != nil {
panic("Failed to read file: " + f)
}
if diff := now.Sub(file.ModTime()); diff < 1 * time.Minute {
print("File " + file.Name() + " has been changed within the last minute\n")
output = true
}
<-semaphore // release
}()
}
wg.Wait()
return output
}

View File

@@ -0,0 +1,115 @@
package webdavupload
import (
"os"
"sync"
"time"
"github.com/studio-b12/gowebdav"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/configloader"
)
// Takes a path on the webdav remote and reads folder contents from it.
func listWebDavFolderContents(config configloader.MDWebDavUploaderConfig, path string) ([]os.FileInfo, error) {
c := gowebdav.NewClient(getWebDavHost(config), config.Username, config.WebDavAuthToken)
c.Connect()
files, err := c.ReadDir(path)
return files, err
}
// Returns a list of the files on the top level of the remote.
func ListTopLevelContents(config configloader.MDWebDavUploaderConfig) ([]os.FileInfo, error) {
return listWebDavFolderContents(config, ".")
}
// Returns a list of the files in the current metadata dir of the remote.
func ListMetadataDir(config configloader.MDWebDavUploaderConfig) ([]os.FileInfo, error) {
return listWebDavFolderContents(config, "./IMPORT_XML")
}
// Returns a list of the files in the current media dir of the remote.
func ListMediaDir(config configloader.MDWebDavUploaderConfig) ([]os.FileInfo, error) {
return listWebDavFolderContents(config, "./IMPORT_IMG")
}
func checkImportConfigExists(c *gowebdav.Client) bool {
_, err := c.Stat("./import_config.txt")
if gowebdav.IsErrNotFound(err) == true {
return false
}
return true
}
// Checks if a remote directory is free for updating:
// This means it can be read, and it does not contain files more recent than
// 10 minutes.
func checkRemoteDirIsFree(c *gowebdav.Client, path string) bool {
now := time.Now()
// Check metadata dir
files, err := c.ReadDir(path)
if err != nil {
print("Failed to load remote contents")
return false
}
for _, f := range(files) {
if diff := now.Sub(f.ModTime()); diff < 10 * time.Minute {
print("File " + f.Name() + " has been recently renamed")
return false
}
}
return true
}
// Checks if either the metadata or the image folder contains files that have
// been changed in the last 10 minutes.
// In this case, it can be assumed that there is a concurrent upload taking place.
func checkRemoteImportFilesAreTooRecent(c *gowebdav.Client) bool {
var wg sync.WaitGroup
var metadataDirFree bool
var mediaDirFree bool
wg.Add(2)
go func() {
defer wg.Done()
metadataDirFree = checkRemoteDirIsFree(c, "./IMPORT_XML")
}()
go func() {
defer wg.Done()
mediaDirFree = checkRemoteDirIsFree(c, "./IMPORT_IMG")
}()
wg.Wait()
if metadataDirFree == true && mediaDirFree == true {
return false
}
return true
}
// Checks the remote for the existence of an import_config.txt.
func CheckRemoteIsFree(c *gowebdav.Client) bool {
if checkImportConfigExists(c) == true {
return false
}
// Check if the remote media and metadata directories currently
// contain very recent files indicating a concurrent upload.
if checkRemoteImportFilesAreTooRecent(c) == true {
return false
}
return true
}

View File

@@ -0,0 +1,35 @@
package webdavupload
import (
"net/url"
"strconv"
"github.com/studio-b12/gowebdav"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/configloader"
)
// Generates the path for a webdav endpoint for the configured instance and institution.
func getWebDavHost(config configloader.MDWebDavUploaderConfig) string {
parsed, err := url.Parse(config.InstanceLink)
if err != nil {
panic("Your settings must be broken. Failed to load instance URL.")
}
output := url.URL{
Scheme: "https",
Host: parsed.Host,
Path: "/musdb/webdav/" + strconv.Itoa(config.InstitutionId),
}
return output.String()
}
// Returns a WebDAV client for the current config.
func GetWebdavClient(config configloader.MDWebDavUploaderConfig) *gowebdav.Client {
c := gowebdav.NewClient(getWebDavHost(config), config.Username, config.WebDavAuthToken)
c.Connect()
return c
}

132
src/webdavupload/Upload.go Normal file
View File

@@ -0,0 +1,132 @@
package webdavupload
import (
"fmt"
"path/filepath"
"os"
"io"
"net/http"
"runtime"
"sync"
"sync/atomic"
"github.com/studio-b12/gowebdav"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/configloader"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/importconfiggen"
)
// Writes an import config to the remote
func SetImportConfigToTheRemote(c *gowebdav.Client, config configloader.MDWebDavUploaderConfig) error {
importConf := importconfiggen.GenerateImportConfig(config)
return c.Write("import_config.txt", []byte(importConf), 0660)
}
// Uploads a list of files to the target folder.
func uploadFiles(c *gowebdav.Client, w io.Writer, files []string, remoteTarget string, outputContext string) {
// Check if the io.Writer is a http writer
_, wImplementsHttpFlusher := interface{}(w).(http.Flusher)
total := len(files)
var counter atomic.Uint64
// Determine the number of upload tasks to be processed concurrently.
// 10 will be a hard maximum to not spam the server.
maxConcTasks := min(10, runtime.NumCPU())
// Set a semaphore to restrict the number of concurrent upload tasks.
semaphore := make(chan struct{}, maxConcTasks)
wg := &sync.WaitGroup{}
fmt.Fprintf(w, "Will upload %v files. Processing %v tasks at a time.\n", total, maxConcTasks)
if wImplementsHttpFlusher == true {
w.(http.Flusher).Flush()
}
for _, f := range(files) {
semaphore <- struct{}{} // acquire
wg.Add(1)
go func() {
defer wg.Done()
basename := filepath.Base(f)
file, fOpenErr := os.Open(f)
if fOpenErr != nil {
panic("Failed to read file: " + f)
}
defer file.Close()
c.WriteStream("./" + remoteTarget + "/" + basename, file, 0644)
counter.Add(1)
fmt.Fprintf(w, "Uploading %d of %d - File: %s (%s)\n", counter, total, basename, outputContext)
if wImplementsHttpFlusher == true {
w.(http.Flusher).Flush()
}
<-semaphore // release
}()
}
wg.Wait()
fmt.Println("Done")
}
// Uploads the selected metadata files.
func UploadMetadataFiles(c *gowebdav.Client, w io.Writer, files []string) {
uploadFiles(c, w, files, "IMPORT_XML", "Uploading metadata files")
}
// Uploads the selected media files.
func UploadMediaFiles(c *gowebdav.Client, w io.Writer, files []string) {
uploadFiles(c, w, files, "IMPORT_IMG", "Uploading media files")
}
// Removes a list of files.
func BatchUnlink(w io.Writer, files []string) {
_, wImplementsHttpFlusher := interface{}(w).(http.Flusher)
maxConcTasks := min(10, runtime.NumCPU())
// Set a semaphore to restrict the number of concurrent upload tasks.
semaphore := make(chan struct{}, maxConcTasks)
wg := &sync.WaitGroup{}
for _, f := range(files) {
semaphore <- struct{}{} // acquire
wg.Add(1)
go func() {
defer wg.Done()
err := os.Remove(f)
if err != nil {
panic("Failed to delete file " + f)
}
fmt.Fprintf(w, "Delete file %s\n", f)
<-semaphore // release
}()
}
wg.Wait()
if wImplementsHttpFlusher == true {
w.(http.Flusher).Flush()
}
}

View File

@@ -0,0 +1,25 @@
<!DOCTYPE html>
<html>
<head>
<title>museum-digital uploader</title>
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta http-equiv="content-type" content="text/html; charset=UTF-8"/>
<link rel="shortcut icon" sizes="16x16 32x32" href="/logo.svg" />
<link rel="shortcut icon" sizes="64x64" href="/logo.svg" />
<link rel="apple-touch-icon" sizes="256x256" href="/logo.svg" />
<meta name="theme-color" content="#FFF" />
<link rel="stylesheet" type="text/css" href="/md-uploader.css" />
</head>
<body>
<header>
<img src="/logo.svg">
<h1>museum-digital:uploader</h1>
<span id="actions"></span>
</header>
<div id="wrap">
<script type='text/javascript' src='/md-uploader.js' async></script>
</body>
</html>

View File

@@ -0,0 +1,423 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg
width="1020"
height="1020"
id="svg2985"
version="1.1"
inkscape:version="1.2 (dc2aedaf03, 2022-05-15)"
sodipodi:docname="logo-md-code-black.svg"
inkscape:export-filename="mdlogo-code-512px.png"
inkscape:export-xdpi="48.189999"
inkscape:export-ydpi="48.189999"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:dc="http://purl.org/dc/elements/1.1/">
<title
id="title42">General logo for museum-digital (black background)</title>
<defs
id="defs2987" />
<sodipodi:namedview
id="base"
pagecolor="#ffffff"
bordercolor="#ffffff"
borderopacity="0.0"
inkscape:pageopacity="0.0"
inkscape:pageshadow="0"
inkscape:zoom="0.98994949"
inkscape:cx="375.27167"
inkscape:cy="658.11439"
inkscape:document-units="px"
inkscape:current-layer="layer1"
showgrid="false"
inkscape:window-width="3840"
inkscape:window-height="2076"
inkscape:window-x="0"
inkscape:window-y="-33"
inkscape:window-maximized="1"
fit-margin-top="5"
fit-margin-left="5"
fit-margin-right="5"
fit-margin-bottom="5"
showguides="false"
inkscape:showpageshadow="2"
inkscape:pagecheckerboard="0"
inkscape:deskcolor="#d1d1d1" />
<metadata
id="metadata2990">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<cc:license
rdf:resource="http://creativecommons.org/licenses/by/4.0/" />
<dc:title>General logo for museum-digital (black background)</dc:title>
<dc:date>2019</dc:date>
<dc:creator>
<cc:Agent>
<dc:title>museum-digital</dc:title>
</cc:Agent>
</dc:creator>
</cc:Work>
<cc:License
rdf:about="http://creativecommons.org/licenses/by/4.0/">
<cc:permits
rdf:resource="http://creativecommons.org/ns#Reproduction" />
<cc:permits
rdf:resource="http://creativecommons.org/ns#Distribution" />
<cc:requires
rdf:resource="http://creativecommons.org/ns#Notice" />
<cc:requires
rdf:resource="http://creativecommons.org/ns#Attribution" />
<cc:permits
rdf:resource="http://creativecommons.org/ns#DerivativeWorks" />
</cc:License>
</rdf:RDF>
</metadata>
<g
inkscape:label="Layer 1"
inkscape:groupmode="layer"
id="layer1"
transform="translate(8,-40.362183)">
<rect
style="fill:#000000;fill-opacity:1;stroke:#000000;stroke-width:8.82220364;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect845"
width="1011.1779"
height="1011.1778"
x="-3.5888982"
y="44.773335"
ry="0" />
<flowRoot
xml:space="preserve"
id="flowRoot3763"
style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;text-align:end;letter-spacing:0px;word-spacing:0px;text-anchor:end;fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-opacity:1"
transform="matrix(1.0379746,0,0,1.0379746,-169.56958,-12.773255)"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997"><flowRegion
id="flowRegion3765"
style="font-family:sans-serif;fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-opacity:1"><rect
id="rect3767"
width="1008.1323"
height="1000.051"
x="-6.0609155"
y="50.290859"
style="font-family:sans-serif;text-align:end;text-anchor:end;fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-opacity:1" /></flowRegion><flowPara
id="flowPara3769"
style="font-size:40px;line-height:1.25;font-family:sans-serif;fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-opacity:1"> </flowPara></flowRoot>
<text
xml:space="preserve"
style="font-style:normal;font-variant:normal;font-stretch:normal;font-size:12.4557px;line-height:0%;font-family:'Bookman Old Style';-inkscape-font-specification:'Bookman Old Style Semi-Light';text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:1.03797;stroke-opacity:1"
x="348.29947"
y="659.89093"
id="text3771"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833"
width="79.41317"
height="79.41317"
x="98.490639"
y="742.45844"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-3"
width="79.413177"
height="79.41317"
x="98.490639"
y="626.96552"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6"
width="79.41317"
height="79.41317"
x="98.490639"
y="511.47241"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:none;fill-opacity:1;stroke:#ffffff;stroke-width:10.21611977;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-3-7"
width="82.178253"
height="47.530357"
x="97.108063"
y="429.24481"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-5"
width="79.413162"
height="79.41317"
x="271.73004"
y="742.45844"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-3-3"
width="79.41317"
height="79.41317"
x="271.73004"
y="626.96552"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5"
width="79.413162"
height="79.41317"
x="271.73004"
y="511.47241"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-5-6"
width="79.413162"
height="79.41317"
x="444.96948"
y="742.45844"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-3-3-2"
width="79.41317"
height="79.41317"
x="444.96948"
y="626.96545"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9"
width="79.413162"
height="79.41317"
x="444.96948"
y="511.47235"
rx="0"
ry="39.706581"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:10.21612072;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-1"
width="47.530357"
height="82.178246"
x="201.05171"
y="510.08987"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:10.21612072;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-1-2"
width="47.530357"
height="82.178246"
x="374.29114"
y="510.08987"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-7"
width="79.413162"
height="79.41317"
x="595.11035"
y="742.45844"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-7-0"
width="79.413162"
height="79.41317"
x="710.60327"
y="742.45844"
rx="0"
ry="0"
inkscape:transform-center-x="-9.9165682"
inkscape:transform-center-y="47.832846"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-7-9"
width="79.413162"
height="79.41317"
x="826.09625"
y="742.45844"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-7-3"
width="79.413162"
height="79.41317"
x="595.11035"
y="626.96552"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-7-3-6"
width="79.413162"
height="79.41317"
x="595.11035"
y="511.47241"
rx="0"
ry="39.706581"
inkscape:transform-center-x="-29.166407"
inkscape:transform-center-y="139.41528"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-7-3-6-0"
width="79.413162"
height="79.41317"
x="710.60327"
y="511.47241"
rx="0"
ry="0"
inkscape:transform-center-x="-29.166375"
inkscape:transform-center-y="139.41528"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:none;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-7-6"
width="79.413162"
height="79.41317"
x="826.09625"
y="626.96552"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-7-3-2"
width="79.413162"
height="79.41317"
x="826.09625"
y="511.47241"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-7-3-6-6"
width="79.413162"
height="79.41317"
x="826.09625"
y="395.97949"
rx="0"
ry="0"
inkscape:transform-center-x="-29.166404"
inkscape:transform-center-y="139.4153"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:12.98119068;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-7-3-6-6-1"
width="79.413162"
height="79.41317"
x="826.09625"
y="280.48651"
rx="0"
ry="0"
inkscape:transform-center-x="-29.166404"
inkscape:transform-center-y="139.41528"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:9.61340618;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-3"
width="41.781116"
height="82.780952"
x="443.28558"
y="509.78845"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:9.61340523;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-75"
width="82.780945"
height="41.78112"
x="443.28558"
y="550.78833"
rx="0"
ry="0"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:9.61340523;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-7-3-6-3"
width="82.780945"
height="41.78112"
x="593.42645"
y="550.78839"
rx="0"
ry="0"
inkscape:transform-center-x="-30.4033"
inkscape:transform-center-y="73.349637"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
<rect
style="fill:#ffffff;fill-opacity:1;stroke:#ffffff;stroke-width:9.61340523;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
id="rect833-6-5-9-7-3-6-5"
width="41.781116"
height="82.780952"
x="634.42627"
y="509.78851"
rx="0"
ry="0"
inkscape:transform-center-x="-15.345135"
inkscape:transform-center-y="145.32765"
inkscape:export-xdpi="96.379997"
inkscape:export-ydpi="96.379997" />
</g>
</svg>

After

Width:  |  Height:  |  Size: 15 KiB

View File

@@ -0,0 +1,183 @@
:root {
--color-bg: #FFF;
--color-bg2: #F5F5F5;
--color-fg: #000;
--color-fg2: #212121;
--color-bg-dark: #212121;
--color-fg-medium: #484848;
--color-borders: #D6D6D6;
--color-accent: #546E7A;
--color-accent-hover: #455A64;
--color-accent-focus: #78909C;
--def-padding-main: 1em;
--def-border-radius: 6px;
}
@media (prefers-color-scheme: dark) {
:root {
--color-bg: #000;
--color-bg2: #2685AF;
--color-fg: #FFF;
--color-fg2: #D6D6D6;
--color-bg-dark: #212121;
--color-fg-medium: #BDBDBD;
--color-borders: #424242;
--color-accent: #546E7A;
--color-accent-hover: #90A4AE;
--color-accent-focus: #78909C;
}
}
html {
background:
linear-gradient(135deg,#0000 18.75%,var(--color-bg2) 0 31.25%,#0000 0),
repeating-linear-gradient(45deg,var(--color-bg2) -6.25% 6.25%,var(--color-bg) 0 18.75%);
background-size: 64px 64px;
color: var(--color-fg);
padding: 0;
font-family: Sans;
}
body { margin: 0; padding: 0; }
header { display: grid; grid-template-columns: auto auto 100fr auto;
align-items: center;
background: var(--color-bg); font-size: 1.5em; padding: .4em 2rem;
box-shadow: 0 4px 2px -2px var(--color-borders); }
header * { margin: 0; padding: 0; vertical-align: middle; }
header img { height: 1em; }
header #actions { grid-column: 4; font-size: .85rem; font-weight: bold; }
header #actions > * { padding: .2em .5em;
border-radius: .2em; color: var(--color-fg2); transition: .2s; }
header #actions > :hover { background: var(--color-accent-hover); color: var(--color-bg); }
h1 { display: inline-block; margin-left: 1em; font-size: .8em; }
#wrap { display: grid;
grid-template-columns: 450px 1fr; grid-template-rows: auto auto;
margin: 1em 2rem; grid-gap: 2em; }
#wrap > * { vertical-align: top; }
main,
aside,
.sec-aside { background: var(--color-bg); border-radius: 1px solid var(--color-borders); box-shadow: 0 0 6px var(--color-borders); border-radius: 2px; }
aside,
.sec-aside { grid-column: 1; }
main { grid-column: 2; grid-row: 1 / span 2; }
h2 { margin: 0; padding: .8rem; font-size: 1.4em;
border-bottom: .15em solid var(--color-borders); }
h3 { margin: 0; padding: .8rem; border-bottom: .15em solid var(--color-borders); }
input { background: var(--color-bg); color: var(--color-fg); }
aside .label-input { display: grid;
grid-template-columns: 150px auto 2em; border-bottom: 1px solid var(--color-borders); }
aside .label-input:hover { border-color: var(--color-accent); }
aside .label-input > * { display: inline-block;
font-size: .9em;
padding: .5rem .8rem; border: 0; outline: 0; }
aside label { font-weight: bold; color: var(--color-fg-medium); }
aside .label-input button { background: none; transition: background .2s; }
aside .label-input button:hover { background: var(--color-accent); }
.icon-help:before { content: " ? "; font-weight: bold; font-size: .8em;
text-align: center; color: var(--color-fg2); transition: background .2s; }
.icon-help { transition: background .2s; }
.icon-help:hover { background: var(--color-accent); }
select { width: 100%; background: var(--color-bg); color: var(--color-fg); }
.updateB { display: block; width: calc(100% - 1em); background: var(--color-bg);
margin: .5em; padding: .5em;
text-align: center;
border: 1px solid var(--color-bg); border-radius: 2px; font-weight: bold;
color: var(--color-accent); transition: background .2s, color .2s;}
.updateB:hover { background: var(--color-accent-hover); color: var(--color-bg); }
.upload-file-list > span { display: block;
padding: .3em .8rem;
font-family: Mono, Courier; font-size: .9em;
border-bottom: 1px solid var(--color-borders); transition: border .2s, background .2s; }
.upload-file-list > span:hover { background: var(--color-bg2); border-color: var(--color-accent-hover); }
.upload-file-list:empty { display: block; position: relative; height: 5em; }
.upload-file-list:empty:before { content: " - - - - - ";
position: absolute; left: 50%; top: 50%; transform: translate(-50%, -50%);
font-weight: bold; font-size: 3em; color: var(--color-borders); }
main { position: relative; }
main h3 { position: sticky; top: 0; background: var(--color-bg); }
.transfer-overlay { display: block;
position: fixed; left: 5em; bottom: 0; width: calc(100% - 10em);
background: var(--color-bg);
border: 2px solid var(--color-accent-hover); border-bottom-width: 0;
box-shadow: 0 0 6px var(--color-borders);
z-index: 10; animation: slide-up .4s ease-out; }
.transfer-overlay h3 { border-bottom: none; }
.transfer-overlay-hl { display: grid; grid-template-columns: auto 1fr auto;
padding-right: .8em;
align-items: center; border-bottom: .15em solid var(--color-borders); }
.transfer-overlay-hl .overlay-close { display: inline-block;
color: var(--color-fg2);
padding: .5em; border-radius: 2px;
grid-column: 3;
transition: .2s; }
.transfer-overlay-hl .overlay-close:hover { background: var(--color-accent-hover); color: var(--color-bg); }
.transfer-overlay-hl .overlay-close:before { display: inline-block; content: " x ";}
.transfer-msgs { font-size: .9em; }
.transfer-msgs > * { display: block; border-bottom: 1px solid var(--color-borders); transition: .2s; }
.transfer-msgs > :hover { border-bottom-color: 1px solid var(--color-accent-hover); }
@keyframes slide-up {
from { transform: translateY(100%); }
to { transform: initial; }
}
iframe { display: block; width: 100%; border: none; height: 60vh; }
.parser-desc { display: block; padding: 0 .8rem;
font-size: .85em; white-space: pre-wrap; }
#wrap.setup-required { grid-template-columns: 1fr 450px 1fr; justify-content: center; }
.setup-required aside { grid-column: 2; width: 450px; margin: auto; }
.setup-required main { display: none; }
/***
* Slider
* Thanks: mrhyddenn
* The following CSS slider is licensed under the MIT license, 2025 mrhyddenn.
* https://uiverse.io/mrhyddenn/old-fish-66
* */
/* The switch - the box around the slider */
.switch { font-size: 17px; position: relative; display: inline-block; width: 62px; height: 35px; transform: scale(0.8); }
/* Hide default HTML checkbox */
.switch input { opacity: 1; width: 0; height: 0;
}
/* The slider */
.slider { position: absolute; cursor: pointer; top: 0; left: 0; right: 0; bottom: 0px;
background: var(--color-bg); transition: .4s; border-radius: 30px; border: 1px solid var(--color-borders);
}
.slider:before { position: absolute; content: "";
height: 1.9em; width: 1.9em; border-radius: 16px; left: 1.2px; top: 0; bottom: 0
background-color: var(--color-bg); box-shadow: 0 2px 5px var(--color-accent-hover); transition: .4s;
}
input:checked + .slider { background-color: var(--color-accent); border: 1px solid transparent;
}
input:checked + .slider:before { transform: translateX(1.5em); background-color: var(--color-accent-focus); }

View File

@@ -0,0 +1,492 @@
"use strict";
class App {
lang;
tls;
config;
parserList;
uploadableFiles;
wrap;
actionsArea;
aside;
main;
generateTextElement(type, value) {
const elem = document.createElement(type);
elem.textContent = value;
return elem;
}
generateLabelInput(id, labelText, explanationText, type, value) {
const output = document.createElement("div");
output.classList.add("label-input");
const label = document.createElement("label");
label.textContent = labelText;
label.setAttribute("for", id);
output.appendChild(label);
const input = document.createElement("input");
input.id = id;
input.type = "text";
input.value = value;
output.appendChild(input);
const help = document.createElement("span");
help.classList.add("icon-help");
help.title = explanationText;
output.appendChild(help);
return {'wrap': output, 'input': input};
}
async generateSidebar() {
this.aside.appendChild(this.generateTextElement("h2", this.tls.settings));
// Set up fields
const instanceLinkArea = this.generateLabelInput("instance-link",
this.tls.instance_link, this.tls.instance_link_explica, "url", this.config.settings.instance);
this.aside.appendChild(instanceLinkArea.wrap);
const usernameArea = this.generateLabelInput("username",
this.tls.username, this.tls.username_explica, "text", this.config.settings.username);
this.aside.appendChild(usernameArea.wrap);
const mailArea = this.generateLabelInput("mail",
this.tls.mail, this.tls.mail_explica, "email", this.config.settings.mail);
this.aside.appendChild(mailArea.wrap);
const institutionIdArea = this.generateLabelInput("institutionId",
this.tls.institution_id, this.tls.institution_id_explica, "number", this.config.settings.institution_id);
this.aside.appendChild(institutionIdArea.wrap);
const tokenArea = this.generateLabelInput("token",
this.tls.token, this.tls.token_explica, "text", this.config.settings.token);
this.aside.appendChild(tokenArea.wrap);
const parserDiv = document.createElement("div");
parserDiv.classList.add("label-input");
const parserLabel = document.createElement("label");
parserLabel.setAttribute("for", "parserSelect");
parserLabel.textContent = this.tls.parser;
parserDiv.appendChild(parserLabel);
const parserSelect = document.createElement("select");
parserSelect.id = "parserSelect";
for (const entry of this.parserList) {
const opt = document.createElement("option");
opt.value = entry.title;
opt.textContent = entry.title;
if (entry.title === this.config.settings.parser) {
opt.setAttribute("selected", "selected");
}
parserSelect.appendChild(opt);
}
parserDiv.appendChild(parserSelect);
const parserHelp = document.createElement("span");
parserHelp.classList.add("icon-help");
parserHelp.title = this.tls.parser_explica;
parserDiv.appendChild(parserHelp);
this.aside.appendChild(parserDiv);
const uploadFolderArea = this.generateLabelInput("upload-folder",
this.tls.upload_directory, this.tls.upload_directory_explica, "text", this.config.settings.upload_directory);
this.aside.appendChild(uploadFolderArea.wrap);
const visibleDivOuter = document.createElement("div");
visibleDivOuter.classList.add("label-input");
const visibleLabel = document.createElement("label");
visibleLabel.setAttribute("for", "visible");
visibleLabel.textContent = this.tls.publish_immediately;
visibleDivOuter.appendChild(visibleLabel);
const visibleDiv = document.createElement("div");
const visibleSwitch = document.createElement("div");
visibleSwitch.classList.add("switch");
const visibleInput = document.createElement("input");
visibleInput.type = "checkbox";
if (this.config.settings.visible === true) {
visibleInput.checked = "checked";
console.log(visibleInput);
}
visibleSwitch.appendChild(visibleInput);
const visibleSlider = document.createElement("span");
visibleSlider.classList.add("slider");
visibleSlider.addEventListener('click', function() {
visibleInput.checked = !visibleInput.checked;
console.log(visibleInput);
});
visibleSwitch.appendChild(visibleSlider);
visibleDiv.appendChild(visibleSwitch);
visibleDivOuter.appendChild(visibleDiv);
const visibleHelp = document.createElement("span");
visibleHelp.classList.add("icon-help");
visibleHelp.title = this.tls.publish_immediately_explica;
visibleDivOuter.appendChild(visibleHelp);
this.aside.appendChild(visibleDivOuter);
// Additional settings
this.aside.appendChild(this.generateTextElement("h3", this.tls.additional_settings));
const addSettingsArea = document.createElement("div");
const app = this;
function generateAdditionalSettingsField(key, value) {
const line = document.createElement("div");
line.classList.add("label-input");
const newKey = document.createElement("input");
newKey.placeholder = app.tls.add_settings_key;
newKey.required = "required";
newKey.value = key;
line.appendChild(newKey);
const newVal = document.createElement("input");
newVal.placeholder = app.tls.add_settings_value;
newVal.required = "required";
newVal.value = value;
line.appendChild(newVal);
const removeB = document.createElement("button");
removeB.textContent = " x ";
removeB.title = app.tls.remove;
line.appendChild(removeB);
removeB.addEventListener('click', function() {
line.parentElement.removeChild(line);
});
return line;
}
if (this.config.settings.settings != undefined && this.config.settings.settings != null && this.config.settings.settings.length != 0) {
for (let settingKey in this.config.settings.settings) {
addSettingsArea.appendChild(generateAdditionalSettingsField(settingKey, this.config.settings.settings[settingKey]));
}
}
this.aside.appendChild(addSettingsArea);
const newSettingLine = document.createElement("form");
newSettingLine.classList.add("label-input");
const newKey = document.createElement("input");
newKey.placeholder = this.tls.add_settings_key;
newKey.required = "required";
newSettingLine.appendChild(newKey);
const newVal = document.createElement("input");
newVal.placeholder = this.tls.add_settings_value;
newVal.required = "required";
newSettingLine.appendChild(newVal);
const newSubmitB = document.createElement("button");
newSubmitB.type = "submit";
newSubmitB.textContent = " > ";
newSubmitB.title = app.tls.add;
newSettingLine.appendChild(newSubmitB);
newSettingLine.addEventListener('submit', function(e) {
e.preventDefault();
e.stopPropagation();
addSettingsArea.appendChild(generateAdditionalSettingsField(newKey.value, newVal.value));
newKey.value = "";
newVal.value = "";
});
this.aside.appendChild(newSettingLine)
const updateB = document.createElement("button");
updateB.textContent = this.tls.save_update;
updateB.classList.add("updateB");
this.aside.appendChild(updateB);
updateB.addEventListener('click', async function() {
let toSave = {
instance: instanceLinkArea.input.value,
username: usernameArea.input.value,
mail: mailArea.input.value,
token: tokenArea.input.value,
institution_id: parseInt(institutionIdArea.input.value),
parser: parserSelect.value,
upload_directory: uploadFolderArea.input.value,
visible: visibleInput.checked,
settings: {}
};
for (const l of addSettingsArea.children) {
toSave.settings[l.children[0].value] = l.children[1].value;
}
const response = await window.fetch('/update-settings', {
method: 'POST', cache: 'no-cache',
credentials: 'same-origin',
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
},
body: "settings=" + encodeURIComponent(JSON.stringify(toSave, null, 2))
});
if (!response.ok) {
console.log("Failed to save");
window.alert("Failed to store.");
}
const json = await response.json();
if (json.success === true) {
location.reload();
}
else {
alert("Error:\n" + json.error);
}
});
}
generateMain() {
this.main.appendChild(this.generateTextElement("h2", this.tls.uploadable_files));
// Metadata
const metadataSec = document.createElement("section");
metadataSec.appendChild(this.generateTextElement("h3", this.tls.metadata_files));
const metadataList = document.createElement("div");
metadataList.classList.add("upload-file-list");
for (const filename of this.uploadableFiles.metadata) {
const fileline = document.createElement("span");
fileline.classList.add("icon-file");
fileline.textContent = filename;
metadataList.appendChild(fileline);
}
metadataSec.appendChild(metadataList);
this.main.appendChild(metadataSec);
// Media
const mediaSec = document.createElement("section");
mediaSec.appendChild(this.generateTextElement("h3", this.tls.media_files));
const mediaList = document.createElement("div");
mediaList.classList.add("upload-file-list");
for (const filename of this.uploadableFiles.media_files) {
const fileline = document.createElement("span");
fileline.classList.add("icon-file");
fileline.textContent = filename;
mediaList.appendChild(fileline);
}
mediaSec.appendChild(mediaList);
this.main.appendChild(mediaSec);
}
manageUpload() {
const transferOverlay = document.createElement("div");
transferOverlay.id = "transfer-overlay";
transferOverlay.classList.add("transfer-overlay");
const hl = document.createElement("div");
hl.classList.add("transfer-overlay-hl");
hl.appendChild(this.generateTextElement("h3", this.tls.upload));
const closeB = document.createElement("span");
closeB.classList.add("overlay-close");
hl.appendChild(closeB);
closeB.addEventListener('click', function() { location.reload(); });
// closeB.addEventListener('click', function() { transferOverlay.parentElement.removeChild(transferOverlay); });
transferOverlay.appendChild(hl);
const transferMsgs = document.createElement("div");
transferMsgs.classList.add("transfer-msgs");
transferOverlay.appendChild(transferMsgs);
this.wrap.appendChild(transferOverlay);
const iframe = document.createElement("iframe");
iframe.src = "/trigger-upload";
transferOverlay.appendChild(iframe);
}
generateActionsMenu() {
if (this.uploadableFiles.metadata.length === 0 && this.uploadableFiles.media_files.length === 0) {
return false;
}
const uploadTrigger = document.createElement("span");
uploadTrigger.textContent = this.tls.upload;
this.actionsArea.appendChild(uploadTrigger);
const app = this;
uploadTrigger.addEventListener('click', function() {
app.manageUpload();
}, {once: true});
// Handle next auto upload
const nextAutoUpload = document.createElement("span");
const startTimeStr = this.tls.next_scheduled_check + ": ";
this.actionsArea.appendChild(nextAutoUpload);
const options1 = {
year: 'numeric',
month: '2-digit',
day: '2-digit',
hour: '2-digit',
minute: '2-digit',
};
const dateTimeFormat1 = new Intl.DateTimeFormat(this.lang, options1);
const nextUploadTime = Date.parse(this.config.next_auto_upload);
const now = new Date();
setInterval(function() {
const diffMs = nextUploadTime - now;
const diffMins = Math.round((diffMs % 86400000) / 60000); // minutes
nextAutoUpload.textContent = startTimeStr + dateTimeFormat1.format(nextUploadTime) + " (in " + diffMins + " mins)";
}, 60);
// Version
const vSpan = document.createElement("a");
vSpan.textContent = this.tls.version + " " + this.config.version;
if (this.config.update_available === true) {
vSpan.style.color = "red";
vSpan.href = "https://gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/releases";
}
this.actionsArea.appendChild(vSpan)
}
// Provides a sidebar entry for describing the parser
generateParserDesc() {
for (const parser of this.parserList) {
if (parser.title == this.config.settings.parser) {
const aside = document.createElement("div");
aside.classList.add("sec-aside");
aside.appendChild(this.generateTextElement("h3", this.tls.parser + ": " + parser.title));
const div = document.createElement("div");
div.classList.add("parser-desc");
const desc = document.createElement("p");
desc.textContent = parser.comment;
div.appendChild(desc);
aside.appendChild(div);
this.wrap.appendChild(aside);
break;
}
}
}
render() {
if (this.config.setup_required === true) {
wrap.classList.add("setup-required");
this.generateSidebar();
}
else {
this.generateSidebar();
this.generateMain();
this.generateActionsMenu();
this.generateParserDesc();
}
}
setupFromApis() {
const app = this;
let done = 0;
function wrapFinish() {
if (done === 4) {
app.render();
}
}
window.fetch('/get-settings', {method: 'GET', cache: 'no-cache', credentials: 'same-origin'})
.then(function(response) { return response.json(); })
.then(function(elements) {
app.config = elements;
done++;
wrapFinish();
});
window.fetch('/list-parsers', {method: 'GET', cache: 'no-cache', credentials: 'same-origin'})
.then(function(response) { return response.json(); })
.then(function(elements) {
app.parserList = elements;
done++;
wrapFinish();
});
window.fetch('/list-uploadable', {method: 'GET', cache: 'no-cache', credentials: 'same-origin'})
.then(function(response) { return response.json(); })
.then(function(elements) {
app.uploadableFiles = elements;
done++;
wrapFinish();
});
window.fetch('/get-translations?lang=' + encodeURIComponent(this.lang), {method: 'GET', cache: 'no-cache', credentials: 'same-origin'})
.then(function(response) { return response.json(); })
.then(function(elements) {
app.tls = elements.webdav_uploader;
done++;
wrapFinish();
});
}
constructor() {
this.wrap = document.getElementById("wrap");
this.lang = navigator.language.substring(0, 2)
if (["de", "en"].includes(this.lang) === false) {
this.lang = "en";
}
this.aside = document.createElement("aside");
this.main = document.createElement("main");
this.actionsArea = document.getElementById("actions");
this.wrap.appendChild(this.aside);
this.wrap.appendChild(this.main);
this.setupFromApis();
}
}
new App();

55
src/webui/headers.go Normal file
View File

@@ -0,0 +1,55 @@
package webui
import (
"net/http"
)
// Sets generally useful HTTP headers, e.g. limiting frame-options.
func setDefaultHeaders(w http.ResponseWriter) {
w.Header().Set("X-Content-Type-Options", "nosniff")
w.Header().Set("X-Frame-Options", "SAMEORIGIN")
w.Header().Set("X-XSS-Protection", "1; mode=block")
w.Header().Set("Access-Control-Allow-Methods", "GET, POST, OPTIONS")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Expose-Headers", "Content-Type")
// w.Header().Set("Strict-Transport-Security", "max-age=63072000")
}
// Sets the HTTP headers expected to be returned when serving HTML pages.
func setHeadersForHtml(w http.ResponseWriter) {
setDefaultHeaders(w)
w.Header().Set("Content-Security-Policy", "default-src 'self'; font-src 'self'; object-src 'none'; frame-src 'self'; frame-ancestors 'none'; base-uri 'none'; script-src 'self'; style-src 'self' 'unsafe-inline'")
}
// Sets the HTTP headers expected to be returned when serving JSON.
func setHeadersForJson(w http.ResponseWriter) {
setDefaultHeaders(w)
w.Header().Set("Content-Type", "application/json")
}
// Sets the HTTP headers expected to be returned when serving CSS files.
func setHeadersForCss(w http.ResponseWriter) {
setDefaultHeaders(w)
w.Header().Set("Content-Type", "text/css")
}
// Sets the HTTP headers expected to be returned when serving JS files.
func setHeadersForJs(w http.ResponseWriter) {
setDefaultHeaders(w)
w.Header().Set("Content-Type", "text/javascript")
}
// Sets the HTTP headers expected to be returned when serving SVG files.
func setHeadersForSvg(w http.ResponseWriter) {
setDefaultHeaders(w)
w.Header().Set("Content-Type", "image/svg+xml")
}
func setHeadersForEventStream(w http.ResponseWriter) {
setDefaultHeaders(w)
w.Header().Set("X-Accel-Buffering", "no")
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
}

View File

@@ -0,0 +1,39 @@
package webui
import (
"encoding/json"
"fmt"
"net/http"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/configloader"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/meta"
)
type getSettingsApiResponse struct {
Version string `json:"version"`
SetupRequired bool `json:"setup_required"`
Settings configloader.MDWebDavUploaderConfig `json:"settings"`
NextAutoUpload string `json:"next_auto_upload"`
UpdateAvailable bool `json:"update_available"`
}
// Generates the API output for listing current settings.
func serveApiGetSettings(w http.ResponseWriter, r *http.Request) {
var nextAutoUpOut string
nextAutoUp, err := scheduledUpload.NextRun()
if err != nil {
nextAutoUpOut = "";
} else {
nextAutoUpOut = nextAutoUp.Format("2006-01-02 15:04:05")
}
setHeadersForJson(w)
output := getSettingsApiResponse{SetupRequired: setupRequired, Settings: config, Version: meta.GetVersion(), NextAutoUpload: nextAutoUpOut, UpdateAvailable: updateAvailable}
outputJson, encodeErr := json.Marshal(output)
if encodeErr != nil {
panic("Failed to create JSON")
}
fmt.Fprint(w, string(outputJson))
}

View File

@@ -0,0 +1,32 @@
package webui
import (
_ "embed"
"fmt"
"net/http"
)
type tlRaw struct {
webdav_uploader map[string]string `json:"webdav_uploader"`
}
//go:embed translation-json/webdav-uploader/de/webdav_uploader.json
var rawTlsDe string
//go:embed translation-json/webdav-uploader/en/webdav_uploader.json
var rawTlsEn string
// Retrieves translation variables in the given language.
func serveApiGetTranslations(w http.ResponseWriter, r *http.Request) {
var rawTls string
if r.URL.Query().Get("lang") == "de" {
rawTls = rawTlsDe
} else {
rawTls = rawTlsEn
}
setHeadersForJson(w)
fmt.Fprint(w, rawTls)
}

View File

@@ -0,0 +1,22 @@
package webui
import (
"encoding/json"
"fmt"
"net/http"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/configloader"
)
// Provides the JSON API for listing all available parsers.
func serveApiListParsers(w http.ResponseWriter, r *http.Request) {
setHeadersForJson(w)
outputJson, encodeErr := json.Marshal(configloader.ListParsers())
if encodeErr != nil {
panic("Failed to create JSON")
}
fmt.Fprint(w, string(outputJson))
}

View File

@@ -0,0 +1,36 @@
package webui
import (
"encoding/json"
"fmt"
"net/http"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/uploadsrcdir"
)
type listUploadableApiResponse struct {
Metadata []string `json:"metadata"`
MediaFiles []string `json:"media_files"`
}
// Provides the JSON API for listing all metadata and media files that
// are currently listed to be uploaded.
func serveApiListUploadable(w http.ResponseWriter, r *http.Request) {
setHeadersForJson(w)
var output listUploadableApiResponse
if setupRequired == true {
output = listUploadableApiResponse{Metadata: []string{}, MediaFiles: []string{}}
} else {
uploadableMetadata, uploadableMedia := uploadsrcdir.GetUploadableFiles(config)
output = listUploadableApiResponse{Metadata: uploadableMetadata, MediaFiles: uploadableMedia}
}
outputJson, encodeErr := json.Marshal(output)
if encodeErr != nil {
panic("Failed to create JSON")
}
fmt.Fprint(w, string(outputJson))
}

View File

@@ -0,0 +1,55 @@
package webui
import (
"fmt"
"net/http"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/uploadsrcdir"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/webdavupload"
)
// Provides the API logic for uploading files.
func serveApiTriggerUpload(w http.ResponseWriter, r *http.Request) {
fmt.Println("Upload requested")
setHeadersForEventStream(w)
w.(http.Flusher).Flush()
fmt.Fprint(w, "Getting list of uploadable files\n")
w.(http.Flusher).Flush()
uploadableMetadata, uploadableMedia := uploadsrcdir.GetUploadableFiles(config)
// If there are no files to upload, do nothing
if len(uploadableMetadata) == 0 && len(uploadableMedia) == 0 {
return
}
fmt.Fprint(w, "Connecting to WebDAV share\n")
w.(http.Flusher).Flush()
c := webdavupload.GetWebdavClient(config)
fmt.Fprint(w, "Connected\n")
w.(http.Flusher).Flush()
if webdavupload.CheckRemoteIsFree(c) == false {
fmt.Println("The remote is currently occupied (very recent files, or an import config is currently waiting to be processed, are present).")
return
}
// Upload the files
if len(uploadableMetadata) > 0 {
fmt.Fprint(w, "Uploading metadata files\n")
w.(http.Flusher).Flush()
webdavupload.UploadMetadataFiles(c, w, uploadableMetadata)
webdavupload.BatchUnlink(w, uploadableMetadata)
}
if len(uploadableMedia) > 0 {
fmt.Fprint(w, "Uploading media files\n")
webdavupload.UploadMediaFiles(c, w, uploadableMedia)
webdavupload.BatchUnlink(w, uploadableMedia)
}
fmt.Fprint(w, "Generating and uploading import configuration\n")
webdavupload.SetImportConfigToTheRemote(c, config)
fmt.Fprint(w, "DONE\n")
w.(http.Flusher).Flush()
}

View File

@@ -0,0 +1,59 @@
package webui
import (
"encoding/json"
"fmt"
"net/http"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/configloader"
)
type updateSettingsResponse struct {
Success bool `json:"success"`
ValidationError string `json:"error"`
}
// Provides the API for updating the general configuration.
func serveApiUpdateSettings(w http.ResponseWriter, r *http.Request) {
setHeadersForJson(w)
// Parse POST data
parseErr := r.ParseForm()
if parseErr != nil {
fmt.Println("Failed to load form data")
return
}
// Load new config from a POST variable 'settings' that provides the full
// settings as a JSON-encoded string.
rawInputConfig := r.PostForm.Get("settings")
var newConfig configloader.MDWebDavUploaderConfig
unmarshalErr := json.Unmarshal([]byte(rawInputConfig), &newConfig)
if unmarshalErr != nil {
fmt.Fprint(w, "{\"success\": false, \"error\": \"Invalid input JSON\"}")
return
}
// Attempt to store the new settings. Validation takes place while storing
var output updateSettingsResponse
validationErr := configloader.StoreConfigToFile(newConfig, "")
if validationErr == nil {
output = updateSettingsResponse{Success: true, ValidationError: ""}
// Reload config
config, setupRequired, _ = configloader.LoadFromFile("")
} else {
output = updateSettingsResponse{Success: false, ValidationError: validationErr.Error()}
}
// Format response in JSON, then print it
outputJson, encodeErr := json.Marshal(output)
if encodeErr != nil {
panic("Failed to create JSON")
}
fmt.Fprint(w, string(outputJson))
}

View File

@@ -0,0 +1,16 @@
package webui
import (
_ "embed"
"fmt"
"net/http"
)
//go:embed assets/index.htm
var indexHtml string
// Serves the app shell. Contents are managed via APIs and JS.
func serveAppShell(w http.ResponseWriter, r *http.Request) {
setHeadersForHtml(w)
fmt.Fprint(w, indexHtml)
}

16
src/webui/serveCss.go Normal file
View File

@@ -0,0 +1,16 @@
package webui
import (
_ "embed"
"fmt"
"net/http"
)
//go:embed assets/md-uploader.css
var webuiCss string
// Serves the style sheet of the web app.
func serveCss(w http.ResponseWriter, r *http.Request) {
setHeadersForCss(w)
fmt.Fprint(w, webuiCss)
}

16
src/webui/serveJs.go Normal file
View File

@@ -0,0 +1,16 @@
package webui
import (
_ "embed"
"fmt"
"net/http"
)
//go:embed assets/md-uploader.js
var webuiJs string
// Serves the JS for the web app.
func serveJs(w http.ResponseWriter, r *http.Request) {
setHeadersForJs(w)
fmt.Fprint(w, webuiJs)
}

16
src/webui/serveLogo.go Normal file
View File

@@ -0,0 +1,16 @@
package webui
import (
_ "embed"
"fmt"
"net/http"
)
//go:embed assets/logo-md-code-black.svg
var logoSvg string
// Serves the logo.
func serveLogo(w http.ResponseWriter, r *http.Request) {
setHeadersForSvg(w)
fmt.Fprint(w, logoSvg)
}

126
src/webui/server.go Normal file
View File

@@ -0,0 +1,126 @@
package webui
import (
"fmt"
"log"
"net/http"
"os/exec"
"runtime"
"time"
"github.com/go-co-op/gocron/v2"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/cli"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/configloader"
"gitea.armuli.eu/museum-digital/museum-digital-webdav-uploader/src/meta"
)
var config configloader.MDWebDavUploaderConfig
var setupRequired bool
var scheduler gocron.Scheduler
var scheduledUpload gocron.Job
var updateAvailable bool
// Opens the application in the default browser.
func openInBrowser(port string) {
var err error
target := "localhost:" + port
switch runtime.GOOS {
case "linux":
err = exec.Command("xdg-open", target).Start()
case "windows":
err = exec.Command("rundll32", "url.dll,FileProtocolHandler", target).Start()
case "darwin":
err = exec.Command("open", target).Start()
default:
err = fmt.Errorf("unsupported platform")
}
if err != nil {
fmt.Println(err)
}
}
func scheduleUploads() {
s, err := gocron.NewScheduler()
defer func() { _ = s.Shutdown() }()
if err != nil {
panic("Failed to set up scheduler")
}
j, jobErr := s.NewJob(
gocron.DurationJob(
3 * time.Hour,
),
gocron.NewTask(
func() {
fmt.Println("Checking if an upload should be done")
fmt.Println(time.Now().Format("20060102150405"))
cli.Upload()
},
),
)
if jobErr != nil {
panic("Failed to set up scheduler job")
}
scheduledUpload = j
s.Start()
select {}
}
// Checks if a new update is available. If that can be confirmed, set updateAvailable
// to true.
func checkUpdateAvailable() {
output, err := meta.CheckVersionIsUpToDate()
// If the RSS stream could not be fetched or parsed, don't say anything
if err != nil || output.Title == "" {
updateAvailable = false
} else {
updateAvailable = true
}
}
// Sets up the server and manages routing.
func Run() {
checkUpdateAvailable()
config, setupRequired, _ = configloader.LoadFromFile("")
// Bind callable URLs
http.HandleFunc("/", serveAppShell) // Serve app shell
http.HandleFunc("/get-settings", serveApiGetSettings) // API for listing current settings
http.HandleFunc("/get-translations", serveApiGetTranslations) // API for listing current settings
http.HandleFunc("/list-uploadable", serveApiListUploadable) // API to list uploadable files
http.HandleFunc("/list-parsers", serveApiListParsers) // API to list available parsers.
http.HandleFunc("/update-settings", serveApiUpdateSettings) // API for updating settings
http.HandleFunc("/trigger-upload", serveApiTriggerUpload) // Serve page for specific files
http.HandleFunc("/md-uploader.css", serveCss) // Main CSS file.
http.HandleFunc("/md-uploader.js", serveJs) // Main JS file.
http.HandleFunc("/logo.svg", serveLogo) // Logo.
port := "8080"
go func() {
// Set port to listen on
fmt.Println("Server started at port " + string(port))
err := http.ListenAndServe(":"+port, nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
}();
go func() {
time.Sleep(5)
openInBrowser(port)
}();
go scheduleUploads()
select{}
}