Comskip with Tv headend

Apollos · November 3, 2017 at 5:35 PM

Quote from helio58

Hi Apollo, hur did you get rid off the error "Segmentation fault (core dumped)(49.00 fps), 1.00 sec(49.00 fps), 0%"

Thanks

You have to just use comskip without an ini file, so your post processor command would simply look like this:

/storage/.kodi/addons/tools.comskip/bin/comskip %f

If anyone figures out a way to get ini files working with this that would be awesome.

andy5macht · November 4, 2017 at 8:36 PM

after copy whole comskip dir (/storage/.kodi/addons/tools.comskip) to:

Code

/storage/.kodi/userdata/addon_data/tool.comskip

i also copied comskip ini to:

Code

/storage/.kodi/userdata/addon_data/tool.comskip/bin/comskip.ini

now i can launch comskip near the ini. but i get the error:

Code

Segmentation fault (core dumped)

Full output:

Display Spoiler

LibreELEC:~/.kodi/userdata/addon_data/tool.comskip/bin # /storage/.kodi/userdata/addon_data/tool.comskip/bin/comskip /var/media/160GB_USB/TV-Record/Reiseticker/Reisetic

ker-sonnenklar.TV2017-11-0421-15.ts

Comskip 0.81.089, made using ffmpeg

Donator build

The commandline used was:

/storage/.kodi/userdata/addon_data/tool.comskip/bin/comskip /var/media/160GB_USB/TV-Record/Reiseticker/Reiseticker-sonnenklar.TV2017-11-0421-15.ts

Using comskip.ini for initiation values.

Mpeg: /var/media/160GB_USB/TV-Record/Reiseticker/Reiseticker-sonnenklar.TV2017-11-0421-15.ts

Exe comskip.exe

Logo: /var/media/160GB_USB/TV-Record/Reiseticker/Reiseticker-sonnenklar.TV2017-11-0421-15.logo.txt

Ini: comskip.ini

Detection Methods to be used:

1) Black Frame

2) Logo - Give up after 2000 seconds

3) Resolution Change

4) Aspect Ratio

ComSkip throttles back from -0001 to -0001.

The time is now 2133 so it's full speed ahead!

Settings

--------

[Main Settings]

;the sum of the values for which kind of frames comskip will consider as possible cutpoints: 1=uniform (black or any other color) frame, 2=logo, 4=scene change, 8=resolution change, 16=closed captions, 32=aspect ration, 64=silence, 255=all.

detect_method=43

;Set to 10 to show a lot of extra info, level 5 is also OK, set to 0 to disable

verbose=10

;Frame not black if any of the pixels of the frame has a brightness greater than this (scale 0 to 255)

max_brightness=60

maxbright=1

;Frame not pure black if a small number of the pixels of the frame has a brightness greater than this. To decide if the frame is truly black, comskip will also check average brightness (scale 0 to 255)

test_brightness=40

;

max_avg_brightness=25

;

max_commercialbreak=600

;

min_commercialbreak=25

;

max_commercial_size=125

;

min_commercial_size=4

;

min_show_segment_length=250

;

max_volume=500

;

max_silence=100

;

non_uniformity=500

[Detailed Settings]

min_silence=12

remove_silent_segments=0

noise_level=5

brightness_jump=200

fps=22

validate_silence=1

validate_uniform=1

validate_scenechange=1

global_threshold=1.05

disable_heuristics=4

[CPU Load Reduction]

thread_count=2

hardware_decode=0

play_nice_start=-1

play_nice_end=-1

play_nice_sleep=2

[Input Correction]

max_repair_size=200

ms_audio_delay=5

volume_slip=40

lowres=0

skip_b_frames=0

[Aspect Ratio]

ar_delta=0.08

cut_on_ar_change=1

[Global Removes]

padding=0

remove_before=0

remove_after=0

added_recording=14

delete_show_after_last_commercial=0

delete_show_before_first_commercial=0

delete_show_before_or_after_current=0

delete_block_after_commercial=0

min_commercial_break_at_start_or_end=39

always_keep_first_seconds=0

always_keep_last_seconds=0

[USA Specific]

intelligent_brightness=0

black_percentile=0.0076

uniform_percentile=0.003

score_percentile=0.71

[Main Scoring]

length_strict_modifier=3

length_nonstrict_modifier=1.5

combined_length_strict_modifier=2

combined_length_nonstrict_modifier=1.25

ar_wrong_modifier=2

excessive_length_modifier=0.01

dark_block_modifier=0.3

min_schange_modifier=0.5

max_schange_modifier=2

logo_present_modifier=0.01

punish_no_logo=1

[Detailed Scoring]

punish=0

reward=0

punish_threshold=1.3

punish_modifier=2

reward_modifier=0.5

[Logo Finding]

border=10

give_up_logo_search=2000

delay_logo_search=0

logo_max_percentage_of_screen=0.12

ticker_tape=0

ticker_tape_percentage=0

ignore_side=0

ignore_left_side=0

ignore_right_side=0

subtitles=0

logo_at_bottom=0

logo_threshold=0.75

logo_percentage_threshold=0.25

logo_filter=0

aggressive_logo_rejection=0

edge_level_threshold=5

edge_radius=2

edge_w*eight=10

edge_step=1

num_logo_buffers=50

use_existing_logo_file=1

two_pass_logo=1

[Logo Interpretation]

connect_blocks_with_logo=1

logo_percentile=0.92

logo_fraction=0.4

shrink_logo=5

shrink_logo_tail=0

before_logo=0

after_logo=0

where_logo=0

min_black_frames_for_break=1

[Closed Captioning]

ccCheck=0

cc_commercial_type_modifier=4

cc_wrong_type_modifier=2

cc_correct_type_modifier=0.75

[Live TV]

live_tv=1

live_tv_retries=4

require_div5=0

div5_tolerance=-1

incommercial_frames=1000

[Output Control]

output_default=1

output_chapters=0

output_plist_cutlist=0

output_zoomplayer_cutlist=0

output_zoomplayer_chapter=0

output_vcf=0

output_vdr=0

output_projectx=0

output_avisynth=0

output_videoredo=0

output_videoredo3=0

videoredo_offset=2

output_btv=0

output_edl=1

output_live=0

edl_offset=0

timeline_repair=1

edl_skip_field=0

output_edlp=0

output_bsplayer=0

output_edlx=0

output_cuttermaran=0

output_mpeg2schnitt=0

output_womble=0

output_mls=0

output_mpgtx=0

output_dvrmstb=0

output_dvrcut=0

output_ipodchap=0

output_framearray=0

output_debugwindow=0

output_tuning=0

output_t*raining=0

output_false=0

output_aspect=0

output_demux=0

output_data=0

output_srt=0

output_smi=0

output_timing=0

output_incommercial=0

output_ffmeta=0

output_ffsplit=0

delete_logo_file=0

cutscene_frame=0

cutscene_threshold=10

windowtitle="Comskip - %s"

cuttermaran_options="cut=\"true\" unattended=\"true\" muxResult=\"false\" snapToCutPoints=\"true\" closeApp=\"true\""

mpeg2schnitt_options="mpeg2schnitt.exe /S /E /R25 /Z %2 %1"

avisynth_options="LoadPlugin(\"MPEG2Dec3.dll\") \nMPEG2Source(\"%s\")\n"

dvrcut_options="dvrcut \"%s.dvr-ms\" \"%s_clean.dvr-ms\" "

[Sage Workarounds]

sage_framenumber_bug=0

sage_minute_bug=0

enable_mencoder_pts=0

[mpeg2video @ 0x1ac2870] Invalid frame dimensions 0x0.

[mpegts @ 0x1abe890] PES packet size mismatch

Input #0, mpegts, from '/var/media/160GB_USB/TV-Record/Reiseticker/Reiseticker-sonnenklar.TV2017-11-0421-15.ts':

Duration: 00:00:53.53, start: 20626.599944, bitrate: 2673 kb/s

Program 24112

Stream #0:0[0x233]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, top first), 544x576 [SAR 32:17 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc

Stream #0:1[0x234](deu): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s (clean effects)

Stream #0:2[0x238](deu): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)

[mpeg2video @ 0x1ac2350] gray decoding requested but not enabled at configuration time

Initial pts = 0.560

Frame Rate set to 25.000 f/s

Format changed to [544 : 576]

Frame: 1 Ratio: 0.95 MinY: 1 MaxY: 576 MinX: 1 MaxX: 544

Resizing scene change array to accomodate 2000 frames.

0:00:49 - 1245 frames in 6.97 sec(178.62 fps), 1.00 sec(183.00 fps), 94%

Looking For Logo in frames 25 to 1250.

Edge count - 25584 Percentage of screen - 83.23% TOO BIG, CAN'T BE A LOGO.

Looking For Logo in frames 50 to 1275.

Edge count - 26013 Percentage of screen - 83.23% TOO BIG, CAN'T BE A LOGO.

Looking For Logo in frames 75 to 1300.

Edge count - 26972 Percentage of screen - 83.23% TOO BIG, CAN'T BE A LOGO.

overread 52

[mpeg2video @ 0x1ac2350] Warning MVs not available

Retry=0 at frame=1306, time= 52.24 seconds

Retry target pos=17831800, pts=1861131032

[mpeg2video @ 0x1b2c650] Invalid frame dimensions 0x0.

Retry t_pos=17831800, l_pos=16942560, t_pts=1861131032, l_pts=1860922795

Strange audio pts step of 49.84309 instead of 0.00000 at frame 1306

Strange video pts step of -1.79950 instead of 0.04050 at frame 1306

overread 52

[mpeg2video @ 0x1b4ed50] Warning MVs not available

Retry=1 at frame=1306, time= 52.24 seconds

Retry target pos=17831800, pts=1861131032

[mpeg2video @ 0x1b2c650] Invalid frame dimensions 0x0.

Retry t_pos=17831800, l_pos=16942560, t_pts=1861131032, l_pts=1860922795

Strange audio pts step of 49.84309 instead of 0.00000 at frame 1306

overread 52

[mpeg2video @ 0x1b4ed50] Warning MVs not available

Retry=2 at frame=1306, time= 52.24 seconds

Retry target pos=17831800, pts=1861131032

[mpeg2video @ 0x1b2c650] Invalid frame dimensions 0x0.

Retry t_pos=17831800, l_pos=16942560, t_pts=1861131032, l_pts=1860922795

Strange audio pts step of 49.84309 instead of 0.00000 at frame 1306

overread 52

[mpeg2video @ 0x1b4ed50] Warning MVs not available

Retry=3 at frame=1306, time= 52.24 seconds

Retry target pos=17831800, pts=1861131032

[mpeg2video @ 0x1b2c650] Invalid frame dimensions 0x0.

Retry t_pos=17831800, l_pos=16942560, t_pts=1861131032, l_pts=1860922795

Strange audio pts step of 49.84309 instead of 0.00000 at frame 1306

overread 52

[mpeg2video @ 0x1b4ed50] Warning MVs not available

1306 frames decoded in 24.14 seconds (54.10 fps)

Parsed 1306 video frames and 1487 audio frames at 54.10 fps

Maximum Volume found is 4804

Finished scanning file. Starting to build Commercial List.

Vol : #Frames

Calculated silence level = 0

Vol : #Frames

Calculated silence level = 0

Vol : #Frames

Calculated silence level = 0

Vol : #Frames

Calculated silence level = 0

Vol : #Frames

Calculated silence level = 0

Logo Histogram - 0.15326

0.000 - 1305 - 0.99923 ********************************************************************************************************************************************************************************************************

0.050 - 0 - 0.99923

0.100 - 0 - 0.99923

0.150 - 0 - 0.99923

0.200 - 0 - 0.99923

0.250 - 0 - 0.99923

0.300 - 0 - 0.99923

0.350 - 0 - 0.99923

0.400 - 0 - 0.99923

0.450 - 0 - 0.99923

0.500 - 0 - 0.99923

0.550 - 0 - 0.99923

0.600 - 0 - 0.99923

0.650 - 0 - 0.99923

0.700 - 0 - 0.99923

0.750 - 0 - 0.99923

0.800 - 0 - 0.99923

0.850 - 0 - 0.99923

0.900 - 0 - 0.99923

0.950 - 0 - 0.99923

Set Logo Quality = 0.77500

Not enough or too much logo's found (0.00), disabling the use of Logo detection

The last ar cblock wasn't closed. Now closing.

Printing AR cblock list before cleaning

-----------------------------------------

Block: 0 Start: 1 End: 1306 AR_R: 0.95 Length: 0:00:52.16, [ 544x 576] minX= 1, minY= 1, maxX=544, maxY=576

After Sorting - 47

--------------

Aspect Ratio 0.95 found on 1300 frames totalling 99.9%

Aspect Ratio 0.97 found on 1 frames totalling 100.0%

After Sorting - 45

--------------

Aspect Ratio 0.95 found on 1306 frames totalling 100.0%

Printing AR cblock list

-----------------------------------------

Block: 0 Start: 1 End: 1306 AR_R: 0.95 Length: 0:00:52.16, [ 544x 576] minX= 1, minY= 1, maxX=544, maxY=576

Segmentation fault (core dumped)

is there something i can try?

Apollos · November 6, 2017 at 3:18 PM

The person who wrote the addon, shanemeagher, compiled it for the pi3 as a favor/request. I do not think he owned or used a pi3 so there was no debuging. My understanding is that ini files simply dont work, and the only alternative is to switch to using one of the TVheadend nightly builds that has it built in and working.

shanemeagher · November 14, 2017 at 12:21 PM

Apollos andy5macht,

I've had a chance to do a little more on my comskip addon. It now has:

- a comskip.ini file included

- option to select from using the included comskip.ini, specify a path to an alternate, or using none

- recent v0.82.003

Here's builds for:

- Generic/x86_64: tools.comskip-8.2.90.zip

- RPi2/arm: tools.comskip-8.2.90.zip

I got an RPi3 recently and the attached (RPi2/arm) seems to run for me although the RPi runs very hot.

Realistically the comskip.ini needs to be adjusted for the channel as the generic one is hit and miss with the channels I use it with.

chewitt CvH

Is it worth submitting to the repo if it'll be packaged with tvheadend in LE9.0? I probably don't have time to commit to maintaining it consistently to be honest but its seem functional at this point.

Thanks,

Shane

ekincaid · November 17, 2017 at 12:14 AM

I am trying to set this up. I am confused. I'm a NOOB at best. I have a RPi3 and am running Libreelec 8.2.0.1

I have installed tools-comskip-8-2-90-zip

Set the following in tvheadend 4.2.3-20

Remove all unsafe characters from filename: YES
Replace whitespace in title with '-': YES
Post-processor command: /storage/.kodi/addons/tools.comskip/bin/comskip %f

Rebooted

I think now if everything is correct I should now have an .edl file also when I record. Am I correct?

I am not seeing that and commercials are not being skipped. What am I missing?

**esenterre** · November 22, 2017 at 8:10 PM

Quote from ekincaid

I am trying to set this up. I am confused. I'm a NOOB at best. I have a RPi3 and am running Libreelec 8.2.0.1

I have installed tools-comskip-8-2-90-zip

Set the following in tvheadend 4.2.3-20

Remove all unsafe characters from filename: YES

Replace whitespace in title with '-': YES

Post-processor command: /storage/.kodi/addons/tools.comskip/bin/comskip %f

Rebooted

I think now if everything is correct I should now have an .edl file also when I record. Am I correct?

I am not seeing that and commercials are not being skipped. What am I missing?

Display More

I'm at the same point as you. The best "setup instructions" I found so far were yours.
Did you manage to make it works ?

Apollos · November 25, 2017 at 5:26 AM

Thanks for the addon update Shanemeagher! Running with the ini files now works great. I couldnt get the ini file to be used by default, but when modifying the post-processor command in TVH like shown below it worked fine.

For you other folks trying to get this to work. Install the zip addon as normal, then in Tvheadend, inter the following in the post processor command line in the recordings tab:

Code

/storage/.kodi/addons/tools.comskip/bin/comskip.bin  --ini=/storage/.kodi/userdata/addon_data/tools.comskip/ini/comskip.ini %f

If you do some googling and want to use other ini files, just replace the existing ini file, or point to a different location of the ini in the command line above.

**CvH** · November 27, 2017 at 8:27 PM Official Post

Quote from shanemeagher

Is it worth submitting to the repo if it'll be packaged with tvheadend in LE9.0?

sorry somehow completely missed it, yes pls pr for master, backport is also possible (after merge in master)

merl · December 12, 2017 at 10:07 PM

Searching for comskip info for a wetek play (imx6.arm) and this thread is the closest I've come. Any chance of an imx6 arm build? I had an old one that was working on an openelec installation but now it just fails with a missing library. CHeers - Merl

marc.bau · December 29, 2017 at 5:52 PM

CvH I have an LE9 B1213 (Generic x64), but comskip is not included. When will it?

**CvH** · December 30, 2017 at 8:34 PM Official Post

Quote from marc.bau

When will it?

I still wait for shanemeagher to pr his changes, idk if he might forgot it or so

RKCRLR · March 23, 2018 at 3:15 PM

I'm a noob running LibreElec 8.2 with Tvheadend 4.2 on a AMLogic S905W box. Has anyone compiled Comskip for an AMLogic ARM box?

Compiling is a bit over my head, I haven't even successfully SSH'd into my box yet (I can get in through Windows file explorer).

Thanks

ozarks · September 4, 2018 at 4:27 PM

I've been playing around and managed to get comskip to work with my AMLogic S912 box.

I'm not sure whether all of the following steps that I used are necessary as is was a case of try this and that until it worked but this is how I achieved it.

1. SSH into the box and create the directory /storage/.kodi/userdata/addon_data/tools.comskip

2. Install the Comskip addon tools.comskip-8.2.90.zip from post #22 for RPi2/arm courtesy of shanemeagher.

3. Fire up TVHeadend on your browser and navigate to Configuration, Recording.

Choose your profile name of choice from the left hand window pane (probably Default profile unless you have created one for specific needs).

Scroll down the options in the right hand pane until you get to Filename Options and check the boxes for Replace whitespace in title with '-': and Remove all unsafe characters from filename:. Also make sure that your chosen profile is Enabled.

Scroll back up a little until you see the option Post-processor command:. In this field enter the text /storage/.kodi/addons/tools.comskip/bin/comskip.bin --ini=/storage/.kodi/userdata/addon_data/tools.comskip/ini/comskip.ini %f

In the left hand pane, click the Save button.

4. Reboot your box,

5. Set-up a timer recording for a programme that will have adverts (note that if you ever stop a recording part way through then post processing will still be carried out).

6. Wait for the recording to finish and then SSH into your box and navigate to your recordings folder. In my case it was /storage/recordings

Navigate your way into the folder containing the recording, named after the show you just recorded.

You will initially see 4 files but don't do anything until post processing is complete and you see the 5th, which is the .edl file.

7. Open the video in Kodi and when the ads begin, the video will skip the ads and you may even see a small pop-up in the bottom right hand side of the Kodi screen with the word Commercial, to indicate that it has reached a point and is skipping it.

Ensure that the .edl file always remains with the video file if you copy it elsewhere.

The other .log and .txt files can be deleted unless you want them for further diagnosis.

A big shout out to all that have contributed and given the answers by their own sheer efforts of trial and error.

I'm using Build: 4.2.5 ~ LibreELEC Tvh-addon v8.2.113 (2017-12-21T20:30:12+0100) of TvHeadend.

Update: I noticed when using this with a current X86/64 build based on Kodi 18 and although the addon would install and there were no errors, there was no post processing.

I then tried an 18 based distro for S912 and it didn't work either, so for now it will only work on Krypton 17.6 builds for AMlogic/Intel.

**CvH** · September 4, 2018 at 7:29 PM Official Post

Just for info, comskip is now also available at the tvheadend addon as soon the pending pr is merged

ozarks · September 4, 2018 at 8:46 PM

Quote from CvH

Just for info, comskip is now also available at the tvheadend addon as soon the pending pr is merged

That's great news.

merl · October 24, 2018 at 9:05 PM

Hi

I had another try with the instructions from ozarcs (thanks!)

I'm on a wetek play box (arm7) so I just tried the pi3/arm version. I tried as per the instructions.

Comments: creating the directory in step 1 is fine but since it is empty then comskip won't find an ini file there.

I tested the post-processor line from the command line, replacing the non-existing ini file with the default one in the installation:

Code

/storage/.kodi/addons/tools.comskip/bin/comskip.bin \
 --ini=/storage/.kodi/addons/tools.comskip/defaults/ini/comskip.ini \
  Modus2018-03-12.ts

This runs, identifies the video file and its metadata, gives with a few error messages then dies:

[mpeg2video @ 0x2c94fb0] Invalid frame dimensions 0x0.

...

[mpeg2video @ 0x2c94ac0] gray decoding requested but not enabled at configuration time

Illegal instruction (core dumped)

I presume running from tvheadend is not going to make any difference.

Any ideas?

ozarks · October 25, 2018 at 5:39 AM

Quote from merl
Hi

I had another try with the instructions from ozarcs (thanks!)

I'm on a wetek play box (arm7) so I just tried the pi3/arm version. I tried as per the instructions.

Comments: creating the directory in step 1 is fine but since it is empty then comskip won't find an ini file there.

I tested the post-processor line from the command line, replacing the non-existing ini file with the default one in the installation:
Code
/storage/.kodi/addons/tools.comskip/bin/comskip.bin \
 --ini=/storage/.kodi/addons/tools.comskip/defaults/ini/comskip.ini \
  Modus2018-03-12.ts
This runs, identifies the video file and its metadata, gives with a few error messages then dies:

[mpeg2video @ 0x2c94fb0] Invalid frame dimensions 0x0.

...

[mpeg2video @ 0x2c94ac0] gray decoding requested but not enabled at configuration time

Illegal instruction (core dumped)

I presume running from tvheadend is not going to make any difference.

Any ideas?
Display More

What I found was that when the addon was installed, the folder that was created was automatically populated with the ini file.

If I am reading things correct, your modification to comskip.ini references a specific file name, which I am guessing won't work unless you can predict the name of the file that will be recorded.

There are also a couple of back slashes in there, which I'm not certain are allowed.

One thing that I did change, which might not have any impact, but worth trying, is to use the default stream profile for Matroska, which is then also selected in the recording section to produce a .MKV file rather than a .TS.

I take it that you are not using a 9.0 based LE release as the addon doesn't work with the latest releases and comskip is now baked into the latest TVheadend builds.

merl · October 25, 2018 at 8:13 AM

Quote from ozarks

What I found was that when the addon was installed, the folder that was created was automatically populated with the ini file.

If I am reading things correct, your modification to comskip.ini references a specific file name, which I am guessing won't work unless you can predict the name of the file that will be recorded.

There are also a couple of back slashes in there, which I'm not certain are allowed.

One thing that I did change, which might not have any impact, but worth trying, is to use the default stream profile for Matroska, which is then also selected in the recording section to produce a .MKV file rather than a .TS.

I take it that you are not using a 9.0 based LE release as the addon doesn't work with the latest releases and comskip is now baked into the latest TVheadend builds.

Thanks for your reply. You are correct I am not on 9.0.

I presumed that tvheadend simply runs the post-processing command where %f refers to the file that was recorded. So I simply tested this on an old file. It reads the file in and correctly parses the metadata.

Doing this should simulate what tvheadend does (unless there is some hidden magic, but I can't imagine what that would be). If I try to run it when pointing to a nonexistent ini file it just terminates immediately with an error message.

(the backslashes are just the standard way to break a command over several lines btw - not important here since I ran it from the command line)

I'll try with mkv...

[EDIT: I see one thing I did wrong here: I installed before I created the folder! Will reinstall and test again!]