Plugins

Plugins let an instance add custom behaviour without forking arcadia and without merging anything upstream. A plugin is a separate service that arcadia calls over HTTP, so it can be written in any language and deployed independently.

For now, plugins can add external sources (scrapers), like the built in TMDB, MusicBrainz, ISBN and Comic Vine ones.

Configuration

Declare your plugins in the scrapers section of config.yml, at the root of the repository:

scrapers:
  - id: anime
    placeholder: Anime url
    sources:
      AniDB:
        - tv_show
      MyAnimeList:
        - tv_show
        - movie
    url: http://anime-plugin:9000/scrape
    timeout_seconds: 30

config.yml is read once at startup. When the scrapers section is absent, no plugin is registered.

Both config.yml and compose.override.yml are git ignored. compose.yml already mounts config.yml into the backend container, so only the plugin services themselves have to be declared in compose.override.yml, which Docker Compose loads automatically on top of compose.yml:

services:
  anime-plugin:
    build: ../anime-plugin
    restart: unless-stopped

Writing a scraper plugin

The declared endpoint is called with the identifier the user typed as the url query parameter:

GET http://anime-plugin:9000/scrape?url=https://anidb.net/anime/1234

It must answer with JSON:

{
  "title_group": { "name": "...", "description": "...", "content_type": "tv_show", "...": "..." },
  "edition_group": null,
  "affiliated_artists": [
    {
      "name": "Some Person",
      "aliases": [],
      "description": "",
      "pictures": ["https://example.com/picture.jpg"],
      "external_links": ["https://example.com/person/1"],
      "roles": ["director"],
      "nickname": null
    }
  ]
}

title_group follows the UserCreatedTitleGroup schema and edition_group the UserCreatedEditionGroup one; both are documented in the OpenAPI specification. All three fields are optional.

Artists are given by name: arcadia creates them, merges the roles of an artist appearing several times, and returns real affiliated artists to the interface. Arcadia also checks beforehand whether a title group already has the submitted link, and appends that link to the scraped title group.

external_links is optional and holds the artist’s pages on public databases and other trackers.

Pictures are rehosted if image rehosting is enabled.

Anything else the plugin needs (API keys, caching, rate limiting) is its own business.

Reporting a failure

A plugin that cannot scrape answers with a status outside the 2xx range and a body holding the message meant for the uploader:

{ "error": "www.example.com answered with 503 Service Unavailable" }

Arcadia shows that message as is, and answers the interface with a 502 whatever status the plugin used. A failure with no such body, an unreachable plugin, and an answer arcadia cannot read are reported as a generic error instead, the details only being logged. Write the message for the uploader: what they can act on (a wrong url, a page holding nothing, a site that is down), never a stack trace or an internal identifier.