[DEV INFO] Grimoire

Posted on | In

Create and use own interconnected metadata for anything.

About

foobar2000, but not for music.

Benefits

  • Create custom interconnected metadata for anything:
    • Actual folders/files on disk (for an annotated/categorized file explorer, catalog of projects/photos/collections/materials)
    • Weblinks (for web browser bookmarks)
    • Concepts (for a comparative table of research keywords)
    • People (for an address/contact book or social graph)
    • ???
  • Rename/move/copy actual folders/files according to this custom metadata.
  • Write custom GUIs and parsers to automatically get/modify some of the metadata.

Save all this into self-hosted graph database.
Or save all this into partitioned .parquet-like file.
Easily edit multiple records at the same time.

Comparison Tables

Base

Q Demo app “Grimoire” from “Educational Archive Dust v5” Grimoire Experimental Release Grimoire
Exists? ✔️ Development completed in 2023 ✔️ Development completed in 2024 ⌛ Development in progress
Purpose ➖ Overcomplicated toy
➕ Practice in small-scale software project management		 ➕ Standalone tool
➕ Practice in advanced Qt development		 ✔️ Professional tool
➕ Practice in modular GUI app architecture
Usable? ✔️ ✔️ 🚧 Partially
Documentation, screenshots, video demos? ✔️Documentation
✔️Video demos		 Click to preview:
Enhanced context menus, table viewsOnly one "playlist" can be active at a timeEnhanced FooEditor with responsive row selectorEnhanced FileRenamer with responsive previews; FooEditor multiple datatype delegate showcaseEnhanced FileRenamer preset management context menuEnhanced FileRenamer GUI with detailed reportsArbitrary relationship editor prototype, compare selected nodesArbitrary relationship editor prototype, explore one nodeArbitrary relationship editor prototype, menu with available functions		 🚧
Pubilshed? ✔️ GitHub repository (click to open) ❌ 🤔 Depends on the developer’s private framework licensing strategy
GUI? ✔️ PyQt5
➖ Fancy CSS (experimental, user-editable)
➖ Overcomplicated
➖ Not user-friendly		 ✔️ PyQt6
➖ Overcomplicated
➕ User-friendly		 ✔️ PyQt6
➕ Simplified
➕ User-friendly
User can define own functionality? ✔️ Overcomplicated plugins architecture
User can mod default behavior, yet the process is cumbersome		 ❌ ✔️ Simplified python API + custom GUI architecture
User is expected to define his own GUIs for any behavior, not covered by default widgets; code is organized into strictly managed catalog of namespaces/endpoints

Graph database features

The app uses the Neo4J database. In this section the word “parameters” means “neo4j node or relationship properties”.

Q Demo app “Grimoire” from “Educational Archive Dust v5” Grimoire Experimental Release Grimoire
Supported datatypes ✔️ String ✔️ Boolean
✔️ Datetime
✔️ Float
✔️ Integer
✔️ String		 ✔️ Boolean
✔️ Datetime
✔️ Dictionary
✔️ Float
✔️ Integer
🤔 List
✔️ String
node explorer/editor1 ✔️ ✔️ ↓ Is replaced by ↓
Custom fields explorer/editor2 ❌ ❌ ✔️
Hardcoded connections explorer/editor3 ❌ ❌ ✔️
Arbitrary connections explorer/editor4 ❌ ✔️ 🤔
GUI database editor ❌ ❌ 🤔
GUI search index editor ❌ ❌ 🤔
GUI like Neo4J graph view ❌ ❌ 🤔

Footnotes:

  1. Can explore only the complete node, returned by the database: MATCH ... RETURN n; can edit its parameters.
  2. Can explore any user-defined query, returned by the database; can edit relevant node/relationship parameters (provided the ID is present): MATCH ... RETURN n3, n1.path, n2.name, r1.role, COUNT(aaa), [ (m1)-[r2:SOME_REL_TYPE]-(m2) | m2 { `ID(m2)`:ID(m2), .title, .path } ] as i_prochiy_trehetazhniy_mat.
  3. Default or user-created GUI automatically assigns connections when adding a node.
  4. User can launch dedicated GUI and manually create/edit/remove any connection or its parameters.

Data manipulation features

The app uses parts of the Kedro framework. In this section the word “parameters” means “kedro node parameters”.

Q Demo app “Grimoire” from “Educational Archive Dust v5” Grimoire Experimental Release Grimoire
GUI to launch / stop / provide place for user-defined inputsparameters and outputs of several Pipelines ❌ ❌ ✔️
GUI to define custom runtimes ❌ ❌ ❌
GUI like Kedro-Viz ❌ ❌ ❌

Partitioned file storage / table editing features

Sometimes it is more convenient to open a table, edit it, save it and forget about it, rather than write nodes to the database.

Q Demo app “Grimoire” from “Educational Archive Dust v5” Grimoire Experimental Release Grimoire
Can save to a partitioned file rather than to the database? ❌ ❌ 🤔
Can act as a general file editor (for example allow to edit any .csv, .parquet, .xls, etc)? ❌ ❌ 🤔

Rationale

There is way too much information one needs to manage. Where was it saved? How was it named? What was it for? Write that down into custom metadata, automatically dump associated files somewhere and search for whatever is needed via easy-to-write self-defined GUI catalogs/visualizations.