Schedules Direct for Tvheadend addon

    You can add as many lineups/channels as you need in schedules direct. I created the Kodi addon interface so people didn't have to use the SSH manual setup to do all of that.

    You should have no problem running LE on the Pi 3 with the backend and frontend of TVH addons enabled. That being said, make sure you have a fast sd card as that impacts the cache performace. Depending on what you want to use for recording, you'll want to make sure that has a fast transfer rate as well. That way if you're recording and playing back at the same time, you don't have a buffering issue.

    I didn't know a thing about python when I started piecing together my addons (zap2xml and sd4tvh) - I just built on others work and looked up whatever I needed to. SD4TVH isn't really a clean build as it is pieced together from two other projects...but it works and makes the schedules direct setup a lot easier for people. Maybe someday I'll have time to build a cleaner version from scratch (or someone else can) but for now - it hopefully get a lot more people to cut the cord!

    If you have any trouble running the addon - let me know!

    Network definitely won't bottleneck here. For recording I have a dedicated storage server that also runs Plex.
    Have a Gb connection directly to a managed switch, then 2Gb trunk to the room where the other switch and media equipment is.

    I will go ahead and try TVH on the same box with this grabber and see how it goes. I might run into the same cache issue with 4 lineups from SD will let you know in a few days.


    I like the SD4 interface, I will eventually have a few suggestions for improvement, but right now I'm still debugging.

    I can confirm that I ran into the unicode errors as well. Although my log looked a little different, it did state "unicode error" in there somewhere.

    I then went ahead and made the above changes, and proceeded to edit the channels from lineup and the problem went away.

    I then wanted to undo the changes and capture all the error logs, but I can't duplicate now. I uninstalled the app, removed the userdata->add_ons->script.module.sd4 directory, reinstalled the app without the above code changes and it works fine??? Not sure why, but I suggest looking into adding those lines to import section as it did help.

    Thanks for testing! Let me know if you have any other issues and in the meantime, I will add those lines to the code.

    Hopefuly you can grab logs next time you have an error. Thanks!

    Edited once, last by edit4ever (March 28, 2017 at 7:32 PM).

    Edit4ever's addons are just fantastic for me. I run both the tvheadend server and client on the same $40 Android box (Amlogic S905) running LibreELEC and his addon for my guide data. Very simple to install and configure.

    I just did a clean install of LE, with TVH 4.2 trying to get the Unicode error to pop up again to capture a log...Unfortunately it ran perfectly fine this time (even with same lineups).
    Seems like an intermediate problem, but I plan to add the Import lines into there anyways just in case.

    Other than that I got it working good and I was impressed as I watched the Pi-3's resources while messing around with the backend software. Only time I ever saw it use a lot was during the actual grab. One CPU core went up to about 100%, and then it was switch cores and another went to 100%, but the over all CPU usage never budged 25% at one given time. I watched the RAM during the grab and out of 735mb available it went up to 194 used but then dropped back down to 184mb used after the grab was over. I'm only doing about 30 channels between the 4 lineups, but I grabbed 14 days.

    Seems fairly stable so far. Nice job!

    Update:

    I too ran out of memory when I tried to download 14 days of EPG.

    I plan to troubleshoot more tonight.

    spydah, how did you increase the available memory for LibreElec?

    Other than this issue, it does seem to be working OK.
    But kind of useless when TVH runs the grabber and I watch all the Free Mem disappear with debug mode on.

    I think I will see if 7 days will work OK for my application and then I will troubleshoot on another Pi3.

    I started to debug to find out where the memory is getting heavily used up.

    The first thing I noticed is when the line

    Code
    schedule_hash_list = self._sd.get_schedule_hash_list(station_ids)

    If the lineup is large say for Digital Cable, then the hash list will be so big that it will consume all or most of the memory.

    the get_schedule_hash_list function inside of schedulesDirect.py returns this as a list. Furthermore, that function calls the API and the API ireturns _post as the list.

    • A major problem here is that the MD5's used for EPG differential are all being dynamically stored into RAM. The more lineups, channels in a lineup, and days requested, the more MD5's get pulled and as a direct result the more RAM get's used. (Less days or filtering list does work here to make it smaller, but not enough)
    • Another problem I saw was that the MD5's are getting downloaded even when day's are set to ZERO.
    • Next if you had enough RAM it then tries to stream out to the xmltv.xml file and this is resource intensive as well. So it might use up the RAM and crash here because the variable "schedule_hash_list" has not released the resources it used until the .py file closes.


    Possible solution: Create a temp working directory. When pulling MD5's from the JSON server, write them to a file instead (would require a little work in the api.py file). Then when comparing the hash just use file open and compare MD5's that way. When job is done, remove all files from the temp working directory.


    I will continue to read through the code and if time persists will see if I can come up with a solution.

    Edited once, last by mudrunner88 (April 10, 2017 at 6:16 PM).

    Thanks for looking into this. I don't actually know much about programming...I just got into this as a hobby and pieced together some work done by other folks to make this work.

    I'm sure there is a much better/cleaner way to produce the addon - but I wanted to help get something quickly working with a kodi interface so others could get an epg up and running easily. My intention was only for over the air signals - as I'm a cord cutter who doesn't like the cable tv model - but obviously there are many people who would like a cable lineup epg, so hopefully there's a way to simplifiy this to work with limited ram usage.

    You know more than you are giving yourself credit for ;) Most people can't come up with their own GUI for Kodi let alone piece together stuff from several authors to make it work.

    Now that you have the core of functionality that you believe is what makes this plugin great, people can grab the code and make suggestions or fork it and give it a makeover. This is what makes open source GREAT!

    Sometimes great ideas come from several minds and not just one. When I took more advanced programming classes it was done with a peer looking over the coders shoulder. The peer would look for syntax errors and try to make suggestions on the fly why doing it a different way would be better. This in turn helps the coder strengthen critical thinking skills, and introduces students to positive criticism (which always shows up with peer reviews of code, or when other people just tinker with someone's code in general).

    So with all that said, I don't enjoy coding, but I'm capable if I get some use out of my time I put in. I medically retired last year and from time to time I need something to tinker with to keep my mind sharp, so I'm more than glad to see what we can do about the RAM issue.

    Edited once, last by mudrunner88 (April 12, 2017 at 3:51 PM).

    Do you know if this would work with TVHeadEnd on RPI3 with network setup as IPTV Auto connected to HDHomeRun Prime? I'm having a real hard time getting any EPG to work with SD with this configuration but can view / see all my channels just fine. Just want a guide to use!
    [hr]
    and my front end (kodi) is installed on my chromebox, and my tvheadend is installed on my RPI3... and this isn't working... :/ bummer... my search continues!
    [hr]
    I think I have it all configured, and it would work, if only I could add EPG Sources to my channels. There are no sources to choose from. I can't figure out why not (is this because I'm using IPTV Auto Network?). I had the same problem with the XMLTV SD JSON grabbers too. All my channels are imported and mapped through a .m3u file under Network / IPTV Auto Network on TVHE v4.1

    Edited once, last by inetjnky (April 13, 2017 at 2:18 PM).

    The addon has a setting for hdhomerun that should load the channels automatically. I don't have an hdhomerun, so I've never been able to test this. If the auto setting doesn't work, you can manually setup the lineup and channels through the addon...then in tvheadend you can link the channels to the schedules direct epg source. You can "subscribe" to a schedules direct account for free for 7 days, which should be enough for you to test out if it works. Here's a basic rundown of what to do:

    • I would create an account on schedules direct
    • Download and install the addon (on the pi)
    • Reboot
    • Log in to the tvheadend web interface and enable the sd4tvh grabber
    • Setup your schedules directusername and password in the config section of the addon
    • Try setting the hdhomerun settings (auto detect or manual ip)
    • You can test if that setting worked by forcing the grabber to run in the tvheadend web interface
    • If that didn't work - disable the hdhomerun setting in the addon config
    • Then run the addon
    • Follow the screen prompts to setup a lineup and add/remove channels (only keep the channels you want a guide for - try just a few to test)
    • Return to the tvheadend web interface and force the grabber to run (or reboot and wait for it to run)
    • Once the grabber runs successfully (monior in the tvh log window) you may need to map the epg source to the channels manually


    Hopefully that gets you started. If you have trouble, just post where you got stuck
    Good luck!.
    [hr]
    One other thing to note - if you were trying to automap the epg source with the m3u file - there was an issue with tvheadend that was just addressed: Bug #4306: EPG source not automatically set - Tvheadend

    That may have been causing your issue with the xmltv sd json grabber - but you still should have been able to manually map the epg source.

    Edited once, last by edit4ever (April 13, 2017 at 3:08 PM).

    Quote from inetjnky


    you may need to map the epg source to the channels manually <-- this is the part that I cannot do. The EPG source is blank. Nothing to select.

    In that case - can you check two things...is there a grabber named sd4tvh that is enabled in the CONFIGURATION > CHANNEL/EPG > EPG GRABBER MODULES tab?

    If so - open the TVheadend log window at the bottom of that tab (press the double arrow in the bottom right)
    Then click the Re-run Internal EPG Grabbers button near the top of the window and see what happens in the log window. (or if you know how to access the log - paste that)

    You should see it run the grabber and after a minute or so it should list a number of channels found. Since you haven't mapped any channels, it won't list any programs - but should list the number of channels that the grab provided. If it looks like it runs in just a couple of seconds - chances are there is an issue with the grab and we will need to see the sd4tvh log that should be in your userdata/addon_data/script.module.sd4tvh folder

    I backed out of all my changes and I am trying to re-install again to start over fresh but I can't find the link you posted before on the actual installation instructions to get this setup. Do you have that link again? I have TVH installed and my m3u file setup and all my channels are being seen, I just am trying to get the EPG setup and need the instructions to get your files installed. Thanks.
    [hr]
    Let me give you my configuration and maybe the installation info will be more helpful...

    I have this addon installed on my chomebox which is my Kodi front end. It's configured.

    I have RPI3 with OSMC installed and TVHeadend as my backend. I'm trying to get this installed on the backend. That's the part that I'm needing some help with.

    Thank you.

    Edited once, last by inetjnky (April 21, 2017 at 8:04 PM).

    I assume you are not running the rpi headless...but if you are, it would be easiest to temporarily connect a monitor. Then download the addon and put it somewhere you can see from your rpi: script.module.sd4tvh-0.2.1.zip

    Then install the addon - I'm used to LE but on OSMC I honk it is under System -> Settings -> Addons -> Install from zip

    Once you have the addon installed you should be able to set its config setting to enter your schedules direct username and password.

    Once you do that, you should be able to run the addon to start the schedules direct lineup management.

    Let me know which step isn't working for you. I'm traveling for a few days, but when I get back if you're still stuck I can try running OSMC on my rpi to see if the steps are different. Or if someone else wants to chime in and test on OSMC I'd appreciate it!

    Forgive me this noobie question, but when you say "install", what exactly should I do?

    I have TVheadend installed on a Xubuntu 16.04 system (Intel x86 in 64bit mode), where I want it to function as a back end. My front end will (probably) be a Rpi3 running OPENelec.

    Google hasn't been much help for TVH add-ons. All the hits seem to involve Kodi (Settings->Configuration->...), not TVH. And since TVH is not running on the Rpi, how could the scripts possibly get installed on OPENelec/Kodi reach the TVH machine where they need to run?

    I've downloaded and unpacked the script.module.sd4tvh file, where there are a few directories like /bin and several .py files. The addon.xml file suggests that there might be a TVH scenario to import script.module.sd4tvh, but I can't find such in the web interface (localhost:9981 on my Xubuntu box).

    Thanks!

    Edited once, last by JMThomas (April 30, 2017 at 2:06 PM).

    Nothing to forgive - we were all noobies once!

    Unfirtunatley, my addon is built for a kodi system running as the backend for tvheadend. Tvheadend doesn' t have addons - but you can add new grabbers to the system in order to pull in guide data. My addon is partially built from the sd2xmltv project - which you can run on ubuntu. You can find it's information here:

    GitHub - astrilchuk/sd2xmltv: A Schedules Direct to xmltv converter

    There is also a grabber that can be run through the xmltv project on ubuntu. That info is here:

    GitHub - kgroeneveld/tv_grab_sd_json: XMLTV grabber for the Schedules Direct JSON API

    For true noobies - I created my kodi addon as running LE on a system and then installing the addon through the kodi interface is the easiest way to get setup and running. It removes all of the hassle for people who are not familiar with the command line.