WebGrab+Plus Preview Releases (OBSOLETE)

    • Official Post

    This thread relates to preview releases of the WebGrab+Plus addon.
    The information it contains is likely outdated and irrelevant to the WebGrab+Plus addon.
    Please refer to and/or update this thread, instead.

    Enjoy!

    Hello,


    I am in the process of creating a WebGrab+Plus addon for LibreELEC, but I have no experience whatsoever with this program.
    I am therefore looking for volunteers to test preview releases of the addon, and provide some feedback.
    I take this opportunity to already thank MikeKL for his valuable contribution.


    The Preview Addons:
    (deleted)


    The Code:
    Pull Request #545


    Addon Home Folder:
    - via kodi: Files, Profile directory, addon_data, service.webgrabplus
    - via smb: libreelec.local//Userdata/addon_data/service.webgrabplus
    - via ssh: at /storage/.kodi/userdata/addon_data/service.webgrabplus


    Configuration:
    Edit the configuration files in the addon home folder.


    Operation:
    The service is run at system startup and subsequently every 6 hours.
    The service produces files guide.xml and mediaportal.xml in the addon home folder.


    To restart the service:
    - via Kodi: disable and enable the addon
    - via ssh: systemctl restart service.webgrabplus


    To inspect operation:
    - via smb: libreelec.local//Userdata/addon_data/service.webgrabplus/WebGrab++.log.txt
    - via ssh: systemctl status service.webgrabplus, or journalctl -u service.webgrabplus


    One shot manual operation via ssh:
    - webgrabplus.start


    Please share your feedback on this thread.


    Enjoy

  • Great work!

    Does this addon work on odroid C2?

  • awiouy, Thanks again for making webgrab++ available, I have now updated my release to that provided in thread

    Notes:

    ---edit---
    Guide created is guide.xml not tvguide.xml (correct in 1st post?)
    ---edit---

    I tried to modify location to /storage/Downloads to make guide visible/selectable via kodi GUI (for selection and use with TV Guide via Kodi GUI) but for some reason guide.xml was not created in expected folder (Webgrab++ appears to have created additional /storage/downloads folder?)

    Hope we can add some basic webgrab++ options modifyable from Kodi GUI webgrab++ settings i.e.

    Guide storage location(s) [local & network]
    Time and Frequency of Webgrab++ run [06:00 Every day, Run Now] (every six hours seems too much?)
    Ability to force a full update run of guide "F" over standard "i" for each channel

    Great to be able to setup in background and forget

    Edited once, last by MikeKL (July 20, 2016 at 7:44 PM).

  • Thank you Marciano!

    This addon (and its mono dependency) should also work on Odroid C2.
    I did however not receive any feedback about it, yet.

    Thanks!

    i Will try this tomorrow on my C2.

    Wich version of webgrab wil be installed?

  • Hi MikeKL,

    As for http://guide.xml/tvguide.xml:
    WebGrab+Plus creates a guide.xml, then
    xmltv_time_correct corrects guide.xml to tvguide.xml... if it can

    So either your chans2correct.xml is empty, or xmltv_time_correct.exe fails for some reason.

    There is not much I can do about this, because the return code of xmltv_time_correct is always 0 , irrespective of wether it succeeds or fails.

    As for accessing http://guide.xml/tvguide.xml from the Kodi GUI:
    They should be accessible from the Kodi GUI via: Profile directory, addon_data, service.webgrabplus.
    Default xml file locations should therefore be fine for local use and polling the guide(s).

    As for guide storage location:
    I will consider this later, if I am able to control xmltv_time_correct.
    Incidentally, I do not intend to program code to edit the xml configuration files.

    As for start time and period:
    Unfortunately, the current implementation is the only one I am confortable with.
    Fortunately, it provides for both users who run LE continuously and users who switch LE off when they do not use it.
    I will let someone more experienced than me find another solution.

    Nevertheless, increasing the period is trivial (is a day ok?), and adding a start now button (ie to trigger webgrabplus.start from the Kodi GUI) should be easy.

    As for "F" and "i" updates:
    How do you trigger "F" or "i" updates from the command line?

    Thank you for your help!

  • awiouy

    Apologies for not responding sooner, everything is working absolutely perfectly, you are an absolute star

    For anyone installing Webgrab++ as LibrELEC Addon (Which takes any pain out of install and having a basic pre-running setup of Webgrab++)

    I would recommend reading jumping right to Webgrab++ Configuration Howto to appreciate power and I believe simplisity of Webgrab++

    OK on to my current setting-up method :-

    Copy required channels.xml & associated .ini into the Userdata/addon_data/service.webgrabplus (for country(s) of interest.
    Modify Webgrab++.config.xml to include the specific channels of interest you wish to grab epg information on.

    If we are running Webgrab++ on same device that we want to view EPG and subsequently watch the channels, Webgrab++ alters any country time zone differences on the fly to be aligned with time zone set on the device

    So then we just read guide.xml output "using a XMLTV reader of choice" after Webgrab++ run is complete. (i.e. Kodi TV Guide addon)

    However, if we plan to transfer guide.xml to a remote device that has a different time zone setting, then timings for channels may/will be misaligned with that particular time zone.
    This is where additional chans2correct.xml comes in if we require it. Basically we just add specific channels that need time zone to be moved forward or back..

    I have already tested chans2correct.xml and as awiouy has already stated an additional tvguide.xml will be created in Userdata/addon_data/service.webgrabplus. so....

    guide.xml typically used on device Webgrab++ is running on and any other devices being transferred to with same time zone setting.
    tvguide.xml for devices running in a different time zone and/or for any problematic channel.xml that webgrab++ has trouble automagically setting correct time zone

    Hope these quick notes help others who want to try-out this preview release of webgrab++ and provide awiouy and the LibreELEC team feedback

    [hr]

    Accessing Guide from Kodi, forgot to ensure hidden files/folder visible in Kodi
    (Many users could miss that, hence request to be able to set a user folder location in GUI settings)

    Understand we don’t want to edit .xml files in Kodi, so easy to edit from a PC across the network to device running LibreELEC anyway.

    Regarding run every six hours philosophy, actually on reflection and a few test runs its not that bad at all, assuming users typically set "i" in prepared .xml files for each channel

    i = incremental i.e. Webgrab++ only updates changes, gaps, repairs and adds new shows (within specified time window to grab) since last run. If no guide.xml exists in expected folder, Webgrab++ creates a new one and then starts from scratch as if nothing had previously existed anyway. (user can delete guide if want a brand new clean guide creating)

    I just not sure its a good idea fixing a time in code that all kodi LibreELEC users could then be hitting same webservices to provide data at similar/same time (just a thought really)

    So around 30 mins for 1st guide creation (in my case) and then with automatic run every six hours It seems that guide updates are typically taking less than 10 mins for around 30 channels from more than one provider.

    Thanks again

    Edited once, last by MikeKL (July 20, 2016 at 7:35 PM).

  • Working on RPi 1 B, LE 17.90.003. Only 4 forehead slaps involved in installation.

    I had to redirect the guide.xml file to /.storage because in my tired state I could not figure out how to navigate iVue settings to the default.

    I will play with it a bit tomorrow, I think the time is off. OOPS, I haven't put a timeserver in this copy yet.

    Edited once, last by donbrew (July 23, 2016 at 3:49 AM).


  • I had to redirect the guide.xml file to /.storage because in my tired state I could not figure out how to navigate iVue settings to the default.

    I has similar, tired state moment as you I forgot to ensure that hidden files and folders were visibile in Kodi settings which is an available option only when in advanced and expert modes.

    i.e. With Krypton and Estuary Interface

    Settings -> Library -> General

    Set "Show hidden Files and Folders" to enabled

    Then you are able to browse to userdata -> addon_data -> servicewebgrabplus folder where the guide.xml is currently being stored by default.

    Edited once, last by MikeKL (July 23, 2016 at 1:22 PM).

  • Got it to work! :) Thanks for linking the guide.

    But how can I get Kodi or TVHeadend to read from the .xml?


  • Got it to work! Thanks for linking the guide.

    But how can I get Kodi or TVHeadend to read from the .xml?

    To be honest, I have not been using external guide.xml with main kodi TV interface and TVheadend, but with for example the TV Guide addon.

    Main reason is I want to maintain full user control of a mixed guide/channel content and links to Live streams from Live TV and/or various addon stream sources.
    (So I can understand what's on when and switch between channels without going backwards and forwards between Kodi Live TV and Streaming addon playback options)

    I am currently using a new fork of original TV Guide which is currently under beta development by Primeaval...thread in Kodi forum can be found HERE

    Wish I could do everything I want within standard kodi TV interface with a single simplified PVR backend, so far I have not found a kodi PVR backend (or combination of backends) that work reliably and suit my particular mixed channel needs.

    I am reasonably sure that latest versions of TVheadend will read an externally provided guide.xml, however I have not tried to set it up and use for myself as I already know it will not suit my specific usage case.

    • Official Post

    Hello!

    I have updated the addon preview, use the appropriate link in the first post to get it.

    What is new:
    - the default configuration comes with a sample (Arte, via Telerama)
    - by default guide.xml is grabbed for two days
    - mdb postprocessing is disabled by default (I could not make it work)
    - guide.xml is not locked while it is updated
    - xmltv_time_correct.exe is applied to guide.xml
    - WG2MP.exe generates mediaportal.xml
    - siteini.pack is updated automatically when the addon is updated

    Basic configuration amounts to adding channels to WebGrab++.config.xml via smb or ssh.

    To pick up the updated configuration files, delete the addon home directory via smb or ssh

    Enjoy!


  • To pick up the updated configuration files, delete the addon home directory via smb or ssh

    awiouy, Please can you clarify....We can install and/or update addon in our existing LibreELEC installation by downloading and then accessing provided .zip in normal way to install an addon.

    If we have already previously setup our webgrab+ configuration files and wish to view new examples from latest zip, we delete our userdata _> addon_data -> servicewebgrabplus folders. Otherwise we can just install from .zip and only underlying installation files are updated, with user configuration items in userdata _> addon_data -> servicewebgrabplus unchanged?

    Thanks for the update

  • How can I make WebGrab write guide.xml to smb:// Windows10machine /EPG/guide.xml ?

    That is the address I can use to get iVue on the RPi to use the guide.xml that I moved there from the RPi. I put that address into the <filename> in WebGrab++.config.xml, but it insists on looking for it in /storage/.kodi/userdata/addon_data/service.webgrabplus/smb:// Windows10machine/EPG/guide.xml.

    When I redirected the guide.xml to /storage/guide.xml it worked fine.

    I am linux dumb, what have I done wrong?

    • Official Post

    awiouy, Please can you clarify....We can install and/or update addon in our existing LibreELEC installation by downloading and then accessing provided .zip in normal way to install an addon.

    If we have already previously setup our webgrab+ configuration files and wish to view new examples from latest zip, we delete our userdata _> addon_data -> servicewebgrabplus folders. Otherwise we can just install from .zip and only underlying installation files are updated, with user configuration items in userdata _> addon_data -> servicewebgrabplus unchanged?

    Thanks for the update

    The update changes the paths of most of the output files, some of which are used by webgrabplus.start
    it is therefore preferable to delete the service.webgrabplus folder.
    If you have added many channels to WebGrab++.config.xml, it may be a good idea to save them someplace, to be able to put them in the new WebGrab++.config.xml.
    I hope I have answered your question.
    [hr]
    [hr]


    How can I make WebGrab write guide.xml to smb:// Windows10machine /EPG/guide.xml ?

    That is the address I can use to get iVue on the RPi to use the guide.xml that I moved there from the RPi. I put that address into the <filename> in WebGrab++.config.xml, but it insists on looking for it in /storage/.kodi/userdata/addon_data/service.webgrabplus/smb:// Windows10machine/EPG/guide.xml.

    When I redirected the guide.xml to /storage/guide.xml it worked fine.

    I am linux dumb, what have I done wrong?


    Why don't you do it the other way around, ie access guide.xml from windows10machine via smb://libreelec.local/userdata/addon_data/service.webgrabplus/guide.xml?


  • [hr]
    [hr]


    Why don't you do it the other way around, ie access guide.xml from windows10machine via smb://libreelec.local/userdata/addon_data/service.webgrabplus/guide.xml?

    I must be denser than you think.
    I typed "smb://libreelec.local/userdata/addon_data/service.webgrabplus/guide.xml" into to Add Sources in Kodi on the Windows machine and got "can't retrieve directory information".

    Earlier I added a source "smb:// 192.168.1.211//storage" I thought that was working, but it only shows the directories, not the files.

    For some reason the sources don't have a problem RPi to Windows. iVue on the RPi can use the guide.xml on the windows machine.

    I know it is something I don't understand.

  • I must be denser than you think.
    I typed "smb://libreelec.local/userdata/addon_data/service.webgrabplus/guide.xml" into to Add Sources in Kodi on the Windows machine and got "can't retrieve directory information".

    Earlier I added a source "smb:// 192.168.1.211//storage" I thought that was working, but it only shows the directories, not the files.

    For some reason the sources don't have a problem RPi to Windows. iVue on the RPi can use the guide.xml on the windows machine.

    I know it is something I don't understand.

    Are you trying to access guide.xml on LibreELEC from Kodi on Windows?
    If this is the case:
    - first, add the corresponding source (ie, a folder, not a file) on Kodi on Windows (files, add source, browse, add network location, etc to add smb://libreelec.local/userdata/addon_data/service.webgrabplus - do not add the file in the source)
    - then, use this source to access files (eg, config.xml) from the addon on Kodi on Windows.
    I hope this answers your question

  • Hi guys, everything worked just as expected.

    I'm in Brasil running a RPi2 with minisatip as backend and IPTV Simple Client as frontend, the receiver I'm using is a DiBcom 8000 ISDB-T. If any one have a similar setup and need some help or have any question please let me know.

    Thank you for your work.