Using cron to download an EPG and/or M3U file?

    I was hoping to find out if it is possible to use a cron job to download a file from a website and store it locally at a certain time. (I barely know what a cron job is.)

    The reason that I can't link to these files directly is because the links are in the format of http://xxx.com/xxx/xxx with no file extension and because the file name constantly changes every time it is downloaded. I will need to specify the download name and have it overwrite the previous file.

    I pieced this together from looking at multiple sources, but it did not work (the 'x's in the cron time represent the test times I used):

    Code
    x x * * * /usr/bin/wget http://xxx.com/xxx/xxx -O /profile/xxx/file.gz

    I saw that there was a Kodi app called Cron for Kodi but I am unsure if 1. my cron is even correct or 2. if I should use this app or something other method. Any help appreciated.

    The OS contains cron already so all you need to do is "crontab -e" to start editing in nano, then paste the cron content, and ctrl+o (save) and ctrl +x (exit) .. and all should be good. NB: In LE /profile doesn't exist so you need to be using somewhere on /storage

    Redirect stdout and stderr output to log file or files.

    Code
    x x * * * /usr/bin/wget http://xxx.com/xxx/xxx -O /storage/xxx/file.gz >/storage/temp.log 2>/storage/temp.err

    Libreelec wget is not regular wget binary. If wget does not work in cron, try curl.

    Code
    /usr/bin/curl http://xxx.com/xxx/xxx > /storage/xxx/file.gz


    "file name constantly changes every time it is downloaded."

    Name can change only if wget refuses to overwrite existing file and creates new one, or that URL is redirector or data generator, which sets output filename in request headers and you are downloading without -O option. wget -O option overrides file name set by server.

    The intent was to download an EPG file from http://m3u4u.com/epg/xxx and point IPTV simple client point to that file locally. (It is not possible for Kodi to access this file directly from the website via remote path due to a 405 error.)

    Code
    x x * * * /usr/bin/curl http://m3u4u.com/xml/xxx > /storage/epg.xml

    I was able to use putty to create a cron job but it did not work and I do not have the skill to troubleshoot the issue. Thanks for the replies but I'm going to find an alternative.

    Quote from pope3909

    Disregard the previous comment as this code worked when using a different website. Thanks!

    You could fake user agent string and referer in curl, if website expects regular browser opening URL and URL being accessed by clicking link on their site.

    Code
    curl http://m3u4u.com/xml/xxx --referer "http://m3u4u.com" --user-agent "Mozilla/5.0 (Windows NT 10.0; rv:109.0) Gecko/20100101 Firefox/115.0"

    Your request would still stand out as it won't have session and other cookies and does not fetch referer page and all its objects.

    Edited once, last by tokul (November 23, 2023 at 5:55 AM).