WebGrab+Plus Preview Releases (OBSOLETE)

Quote from awiouy

Hello,

I am in the process of creating a WebGrab+Plus addon for LibreELEC, but I have no experience whatsoever with this program.
I am therefore looking for volunteers to test preview releases of the addon, and provide some feedback.
I take this opportunity to already thank MikeKL for his valuable contribution.

The Preview Addons:
- for LibreELEC 7.0
- for LibreELEC 8.0

The Code:
Pull Request #545

Configuration:
Edit the configuration files in the addon configuration folder:
- via smb: libreelec.local//Userdata/addon_data/service.webgrabplus
- via ssh: at /storage/.kodi/userdata/addon_data/service.webgrabplus
Make sure that the xml configuration files refer to valid files in the LibreELEC file system.

Operation:
The service is run at system startup and subsequently every 6 hours.
The service produces file tvguide.xml in the addon configuration folder.

To restart the service:
- via Kodi: disable and enable the addon
- via ssh: systemctl restart service.webgrabplus

To inspect operation via ssh:
- journalctl -u service.webgrabplus
- systemctl status service.webgrabplus

One shot manual operation via ssh:
- webgrabplus.start

Please share your feedback on this thread.

Enjoy

Display More

Great work!

Does this addon work on odroid C2?

awiouy, Thanks again for making webgrab++ available, I have now updated my release to that provided in thread

Notes:

---edit---
Guide created is guide.xml not tvguide.xml (correct in 1st post?)
---edit---

I tried to modify location to /storage/Downloads to make guide visible/selectable via kodi GUI (for selection and use with TV Guide via Kodi GUI) but for some reason guide.xml was not created in expected folder (Webgrab++ appears to have created additional /storage/downloads folder?)

Hope we can add some basic webgrab++ options modifyable from Kodi GUI webgrab++ settings i.e.

Guide storage location(s) [local & network]
Time and Frequency of Webgrab++ run [06:00 Every day, Run Now] (every six hours seems too much?)
Ability to force a full update run of guide "F" over standard "i" for each channel

Great to be able to setup in background and forget

Quote from awiouy

Thank you Marciano!

This addon (and its mono dependency) should also work on Odroid C2.
I did however not receive any feedback about it, yet.

Thanks!

i Will try this tomorrow on my C2.

Wich version of webgrab wil be installed?

awiouy

Apologies for not responding sooner, everything is working absolutely perfectly, you are an absolute star

For anyone installing Webgrab++ as LibrELEC Addon (Which takes any pain out of install and having a basic pre-running setup of Webgrab++)

I would recommend reading jumping right to Webgrab++ Configuration Howto to appreciate power and I believe simplisity of Webgrab++

OK on to my current setting-up method :-

Copy required channels.xml & associated .ini into the Userdata/addon_data/service.webgrabplus (for country(s) of interest.
Modify Webgrab++.config.xml to include the specific channels of interest you wish to grab epg information on.

If we are running Webgrab++ on same device that we want to view EPG and subsequently watch the channels, Webgrab++ alters any country time zone differences on the fly to be aligned with time zone set on the device

So then we just read guide.xml output "using a XMLTV reader of choice" after Webgrab++ run is complete. (i.e. Kodi TV Guide addon)

However, if we plan to transfer guide.xml to a remote device that has a different time zone setting, then timings for channels may/will be misaligned with that particular time zone.
This is where additional chans2correct.xml comes in if we require it. Basically we just add specific channels that need time zone to be moved forward or back..

I have already tested chans2correct.xml and as awiouy has already stated an additional tvguide.xml will be created in Userdata/addon_data/service.webgrabplus. so....

guide.xml typically used on device Webgrab++ is running on and any other devices being transferred to with same time zone setting.
tvguide.xml for devices running in a different time zone and/or for any problematic channel.xml that webgrab++ has trouble automagically setting correct time zone

Hope these quick notes help others who want to try-out this preview release of webgrab++ and provide awiouy and the LibreELEC team feedback

[hr]

Quote from awiouy

As for accessing http://guide.xml/tvguide.xml from the Kodi GUI:
They should be accessible from the Kodi GUI via: Profile directory, addon_data, service.webgrabplus.
Default xml file locations should therefore be fine for local use and polling the guide(s).

As for guide storage location:
I will consider this later, if I am able to control xmltv_time_correct.
Incidentally, I do not intend to program code to edit the xml configuration files.

As for start time and period:
Unfortunately, the current implementation is the only one I am confortable with.
Fortunately, it provides for both users who run LE continuously and users who switch LE off when they do not use it.
I will let someone more experienced than me find another solution.

Nevertheless, increasing the period is trivial (is a day ok?), and adding a start now button (ie to trigger webgrabplus.start from the Kodi GUI) should be easy.

As for "F" and "i" updates:
How do you trigger "F" or "i" updates from the command line?

Thank you for your help!

Display More

Accessing Guide from Kodi, forgot to ensure hidden files/folder visible in Kodi
(Many users could miss that, hence request to be able to set a user folder location in GUI settings)

Understand we don’t want to edit .xml files in Kodi, so easy to edit from a PC across the network to device running LibreELEC anyway.

Regarding run every six hours philosophy, actually on reflection and a few test runs its not that bad at all, assuming users typically set "i" in prepared .xml files for each channel

i = incremental i.e. Webgrab++ only updates changes, gaps, repairs and adds new shows (within specified time window to grab) since last run. If no guide.xml exists in expected folder, Webgrab++ creates a new one and then starts from scratch as if nothing had previously existed anyway. (user can delete guide if want a brand new clean guide creating)

I just not sure its a good idea fixing a time in code that all kodi LibreELEC users could then be hitting same webservices to provide data at similar/same time (just a thought really)

So around 30 mins for 1st guide creation (in my case) and then with automatic run every six hours It seems that guide updates are typically taking less than 10 mins for around 30 channels from more than one provider.

Thanks again

Working on RPi 1 B, LE 17.90.003. Only 4 forehead slaps involved in installation.

I had to redirect the guide.xml file to /.storage because in my tired state I could not figure out how to navigate iVue settings to the default.

I will play with it a bit tomorrow, I think the time is off. OOPS, I haven't put a timeserver in this copy yet.

Quote from donbrew

I had to redirect the guide.xml file to /.storage because in my tired state I could not figure out how to navigate iVue settings to the default.

I has similar, tired state moment as you I forgot to ensure that hidden files and folders were visibile in Kodi settings which is an available option only when in advanced and expert modes.

i.e. With Krypton and Estuary Interface

Settings -> Library -> General

Set "Show hidden Files and Folders" to enabled

Then you are able to browse to userdata -> addon_data -> servicewebgrabplus folder where the guide.xml is currently being stored by default.

Quote from MikeKL

awiouy

Apologies for not responding sooner, everything is working absolutely perfectly, you are an absolute star

For anyone installing Webgrab++ as LibrELEC Addon (Which takes any pain out of install and having a basic pre-running setup of Webgrab++)

I would recommend reading jumping right to Webgrab++ Configuration Howto to appreciate power and I believe simplisity of Webgrab++

OK on to my current setting-up method :-

Copy required channels.xml & associated .ini into the Userdata/addon_data/service.webgrabplus (for country(s) of interest.
Modify Webgrab++.config.xml to include the specific channels of interest you wish to grab epg information on.

If we are running Webgrab++ on same device that we want to view EPG and subsequently watch the channels, Webgrab++ alters any country time zone differences on the fly to be aligned with time zone set on the device

So then we just read guide.xml output "using a XMLTV reader of choice" after Webgrab++ run is complete. (i.e. Kodi TV Guide addon)

However, if we plan to transfer guide.xml to a remote device that has a different time zone setting, then timings for channels may/will be misaligned with that particular time zone.
This is where additional chans2correct.xml comes in if we require it. Basically we just add specific channels that need time zone to be moved forward or back..

I have already tested chans2correct.xml and as awiouy has already stated an additional tvguide.xml will be created in Userdata/addon_data/service.webgrabplus. so....

guide.xml typically used on device Webgrab++ is running on and any other devices being transferred to with same time zone setting.
tvguide.xml for devices running in a different time zone and/or for any problematic channel.xml that webgrab++ has trouble automagically setting correct time zone

Hope these quick notes help others who want to try-out this preview release of webgrab++ and provide awiouy and the LibreELEC team feedback

[hr]

Accessing Guide from Kodi, forgot to ensure hidden files/folder visible in Kodi
(Many users could miss that, hence request to be able to set a user folder location in GUI settings)

Understand we don’t want to edit .xml files in Kodi, so easy to edit from a PC across the network to device running LibreELEC anyway.

Regarding run every six hours philosophy, actually on reflection and a few test runs its not that bad at all, assuming users typically set "i" in prepared .xml files for each channel

i = incremental i.e. Webgrab++ only updates changes, gaps, repairs and adds new shows (within specified time window to grab) since last run. If no guide.xml exists in expected folder, Webgrab++ creates a new one and then starts from scratch as if nothing had previously existed anyway. (user can delete guide if want a brand new clean guide creating)

I just not sure its a good idea fixing a time in code that all kodi LibreELEC users could then be hitting same webservices to provide data at similar/same time (just a thought really)

So around 30 mins for 1st guide creation (in my case) and then with automatic run every six hours It seems that guide updates are typically taking less than 10 mins for around 30 channels from more than one provider.

Thanks again

Display More

Got it to work! Thanks for linking the guide.

But how can I get Kodi or TVHeadend to read from the .xml?

Quote from eikum

Got it to work! Thanks for linking the guide.

But how can I get Kodi or TVHeadend to read from the .xml?

To be honest, I have not been using external guide.xml with main kodi TV interface and TVheadend, but with for example the TV Guide addon.

Main reason is I want to maintain full user control of a mixed guide/channel content and links to Live streams from Live TV and/or various addon stream sources.
(So I can understand what's on when and switch between channels without going backwards and forwards between Kodi Live TV and Streaming addon playback options)

I am currently using a new fork of original TV Guide which is currently under beta development by Primeaval...thread in Kodi forum can be found HERE

Wish I could do everything I want within standard kodi TV interface with a single simplified PVR backend, so far I have not found a kodi PVR backend (or combination of backends) that work reliably and suit my particular mixed channel needs.

I am reasonably sure that latest versions of TVheadend will read an externally provided guide.xml, however I have not tried to set it up and use for myself as I already know it will not suit my specific usage case.

Quote from awiouy

To pick up the updated configuration files, delete the addon home directory via smb or ssh

awiouy, Please can you clarify....We can install and/or update addon in our existing LibreELEC installation by downloading and then accessing provided .zip in normal way to install an addon.

If we have already previously setup our webgrab+ configuration files and wish to view new examples from latest zip, we delete our userdata _> addon_data -> servicewebgrabplus folders. Otherwise we can just install from .zip and only underlying installation files are updated, with user configuration items in userdata _> addon_data -> servicewebgrabplus unchanged?

Thanks for the update

How can I make WebGrab write guide.xml to smb:// Windows10machine /EPG/guide.xml ?

That is the address I can use to get iVue on the RPi to use the guide.xml that I moved there from the RPi. I put that address into the <filename> in WebGrab++.config.xml, but it insists on looking for it in /storage/.kodi/userdata/addon_data/service.webgrabplus/smb:// Windows10machine/EPG/guide.xml.

When I redirected the guide.xml to /storage/guide.xml it worked fine.

I am linux dumb, what have I done wrong?

Quote from awiouy

[hr]
[hr]

Why don't you do it the other way around, ie access guide.xml from windows10machine via smb://libreelec.local/userdata/addon_data/service.webgrabplus/guide.xml?

I must be denser than you think.
I typed "smb://libreelec.local/userdata/addon_data/service.webgrabplus/guide.xml" into to Add Sources in Kodi on the Windows machine and got "can't retrieve directory information".

Earlier I added a source "smb:// 192.168.1.211//storage" I thought that was working, but it only shows the directories, not the files.

For some reason the sources don't have a problem RPi to Windows. iVue on the RPi can use the guide.xml on the windows machine.

I know it is something I don't understand.

Hi guys, everything worked just as expected.

I'm in Brasil running a RPi2 with minisatip as backend and IPTV Simple Client as frontend, the receiver I'm using is a DiBcom 8000 ISDB-T. If any one have a similar setup and need some help or have any question please let me know.

Thank you for your work.