Quantcast
Channel: Recent Discussions — Plex Forums
Viewing all articles
Browse latest Browse all 151235

[Release] HTTP Anidb Metadata Agent (HAMA)

$
0
0

I am officially forking the HTTP Anidb Metadata Agent (HAMA) made by Atomicstraberry and posted here: http://forums.plexapp.com/index.php/topic/66918-release-http-anidb-metadata-agent-hama/After having concerted with him by mail, i will take over but give due feedback, as he created it, and i would and could not have done it without his code first, as i didn't know Python before. Great language by hte way!!!
As discussed with him:

  • His name will always be part of the header as he created it and was kind enough to let me take over
  • I put the source on GIT: https://github.com/ZeroQI/Hama . i am new to git, so please be understanding...
  • I create a new thread to be able to control the new releases on this first post [this one]

The readme is located here:https://github.com/ZeroQI/Hama/blob/master/README.mdIt will indicate the changes since v0.4Major changes:

  • Title search 100% local using AniDB title database XML file.
  • TVDB and AniDB XML serie page download cached a week by default
  • TVDB Poster by default, downloading AniDB unique poster only if no TVDB poster present
  • ScudLee XML mapping file to convert AniDB id to TVDB id without TVDB search needed (removed from )
  • Studio when missing in AniDB
  • Collection for Movie sets from XML mapping file (and matching with AniDB related anime fields)
  • Using episode summary from TVDB when present
  • HTML links to the TVDB and AniDB in Serie and episode page (off by default)
  • Warning messages for community improvement (missing anidb poster, missing TVDB en poster, missing episode summary, studio variable in AniDB AND mappingfile, theTVDB id missing from mapping file)
  • Themes songs 30s, from plex server using tvdb id [http://wiki.plexapp.com/index.php/TV_Themes]

This should prevent AniDB banning, since no search is done, no poster is downloaded, just the serie XML by ID.I will modify this post with any new releases, i am polishing out the bugs for now:

  • Testing speed and accuracy. need information on performance
  • Tweaking HTML links when TVDB id not in list, alerts when episode summary empty, etc...
  • Test serie/movie within both serie and movie folder settings

You can browse the source code and replace it in place of v0.4 as a beta, but the XML files need to be in Resources folder at the same level as code one

 See github readme, that's the most recent one

How it works

==========

Scanners show the files or not in plex. if it's not showing, it's the scanner fault

Agent when files are showing update the metadata. This agent:

   . match the title to the AniDB Database locally (no online query to AnidDB to prevent getting banned for a day). It use synonyms, and search titles in all languages. you can select the language order in plugin settings

   . use xml matching table to fing the TVDB serie ID.

   . use posters from ANiDB (off by default, because they are crap but the only source for Hentai posters) or by default TVDB (use high res constant resolution posters, but do not store Hentai, have fanart and seasons posters as well)

   . The XML mapping file comes from the XBMC AniDB mod plugin and all xmls are cached 2 weeks and downloaded automatically

   . Use TheTVDB for Anime Episode summaries (AniDB seem to not use summaries for episodes)

   . Use theme songs from Plex server (backlog of 2 years, no need to update themes for now) indexed on thetvdb id. I store them locally using the anidb id if you want to create your own packages (and share?) and therefore you can have the right season music theme song...

   . logs like missing episodes are generated, also contain links to update episode summaries and missing posters... Give back digitally to the community...

The XMLs are downloaded (cached for 2 weeks) and a copy is saved here and used in case of internet issue
 
   . anime-titles.xml, the AniDB title database, part of the AniDB HTTP API : http://anidb.net/api...e-titles.xml.gz decompressed (xml extension). this is parsed locally to find serie title matches
 
   . anime-list-master.xml   ScudLee's AniDB to TVDB xml mapping file, give studio and episode mapping list for te episode overview, downloaded from https://raw.github.c...list-master.xml
 
   . anime-movieset-list.xml ScudLee's movie collection (Because XBMC only support movie collection and the files were develloped for AniDB mod XBMC plugin), downloaded from https://raw.github.c...vieset-list.xml
 
Agents can only write data in data folder as binary objects or as dictionaries, but cannot create folders unfortunately (if i am wrong, let me know)
I use theses folders to cache all pictures, theme songs, since they are not cached by plex.
This way, even if you recreate the whole Plex anime folder entry, you do not have to download the same file again.

Known Issues:
===========
 
People names in Kanji

Producers and actors names are using romaji name like anidb does. I have been requested for an option in Kanji
Unfortunatly in AniDB serie XML (example: http://api.anidb.net...over=1&aid=2656), the creator names are like that:
<creators>
<name id="735" type="Animation Work">Gonzo</name>
<name id="1793" type="Character Design">Iijima Hiroya</name>
<name id="102" type="Direction">Kizaki Fuminori</name>
<name id="1793" type="Chief Animation Direction">Iijima Hiroya</name>
<name id="22332" type="Music">The RZA</name>
<name id="11136" type="Original Work">Okazaki Takashi</name>
</creators>
I would have to process one more XML [http://anidb.net/per...&creatorid=6692] to translate it in kanji, which means one more access to AniDB for each. I could create a local (or remote web) cache but for now that would require too much coding at the time. if an xml existed out of anidb with the name i would probably use it
 
Series with year in title

Plex remove the year from the scanner side before Hama receive it, i need to check if i could correct it by modding the scanner not to drop the year
 
Hentai with no poster?

TVDB doesn't list sport and porn so hentai will not be listed there, so no fanart, no banner, no (season) posters and no episode summary. The same happens if no tvdb entry is there, or if not in the mapping file yet.
 
Movies not being detected

You did use the serie agent for a movie didn't you ?
It's not showing so it's a scanner issue you dope.... Just put 01 after the movie name and it will be picked up and yes my wonderful agent manage this case, but will gather only serie type of information (no producer)
 
Test series name
================
   . Semi-colon: Chaos;Head, Steins;Gate, Myself;Yourself
   . Year: KissXSis (2010), Black Rock Shooter (2012), Fate Zero (2012), Seitokai Yakuindomo (2011), Tenjou Tenge (2005)
   . Exclamation mark: K-On!, K-On!!
   . Apostrophe: Seitokai Yakuindomo'
   . Love Live 2
   . Terra e..
   . Movies: Gekijouban Shakugan no Shana, Kowarekake no Orgel, Sora no Otoshimono Project Pink
   . Strange" has no match and no semicolon to blame
   . "Toaru Hikuushi e no Koiuta" had no tvdb match like "Onee-chan ga kita", "Jigoku Sensei Nube" and "Corpse Party Tortured Souls".
 
 
To Do
=====
   . Package of Studio Logos (see post here). Will not work on that but somebody else can
   . Package of Theme Songs supported, but none exist (name convention: anidbid.mp3). Plex use 30s songs but use seasons, so a package of songs capped at 30s should share the same legality. will not work on that but local loading works
   . Get Plex to not crash when writing to Agent Data folders, but even with try except, it fails if folders not created already so outside my reach and plex has to fix

History

======

This in no longer accurate, missing all last year imrovements...

   . 2014-05-26

         - added better folder management for XMLs files

         - added support when data folders are missing

         - added clear-cache command for serie title to clear the cache

 

[...]

   . 2015-04-07

         - Scanner: season folders can include serie name (tvdb naming convention)

         - Scanner: removed parenthesis "()" and brackets ([], {}) groups in filename to increase recognition rate

         - Scanner: manage better multi-episodes (s01e01-02 weren't supported)

         - Scanner: added format Byousoku 5 Centimeter - 1 - The Chosen Cherry Blossoms - [RAW](3d312152) [Thx TiS]

         - Agent: removed bug crashing when op/ed present with no number

         - Agent: producer, writers code was broken for both movies and series, fixed [thx -MaNIaC-]

 

   . 2015-04-08

         - Agent: removed bug crashing with Anidb specials with anidb number and letter convention

 

   . 2015-04-11

         - Agent: series with absolute numbering  in tvdb will now have summaries


Viewing all articles
Browse latest Browse all 151235

Trending Articles