Having a few issues and need help!

[joshuabolick]

Hello,

Mainly two quesitons, we are using PySceneDetect to detect scene changes in old VHS footage and I am having trouble getting the using of the proper combination of detectors and settings for those detectors to get it to always detect any solid color frames, static, rolling frames / vertical bars, and any other VHS artifacts. I have tried a lot of detector combinations based on reading the documentation and so far it will always miss some section of static or solid color or other VHS artifacts even if I set the detector settings to be super sensitive.

What are the best detectors and ideal settings for those detectors to achieve this?

Also using PySceneDetect should the frame count start at 0 or 1? I have been using it starting it at 0 which makes sense to me but we are integrating with another older system which I am not sure exactly how all of that is handled. For example on one video it detects a new scene cut at a certain frame, which is all black, but then the very next frame is the actual good content that I want, but I could not seem to find the right detectors / settings to use to get that to be right and give me the cut at the start of the good content frame, not the all black frame right before it.

Any help would be greatly appreciated thank you!!

[Breakthrough]

Mainly two quesitons, we are using PySceneDetect to detect scene changes in old VHS footage and I am having trouble getting the using of the proper combination of detectors and settings for those detectors to get it to always detect any solid color frames, static, rolling frames / vertical bars, and any other VHS artifacts. I have tried a lot of detector combinations based on reading the documentation and so far it will always miss some section of static or solid color or other VHS artifacts even if I set the detector settings to be super sensitive.

What combinations have you tried so far? Are there any samples you can share?

What are the best detectors and ideal settings for those detectors to achieve this?

You can find benchmark scores for each detector at https://github.com/Breakthrough/PySceneDetect/tree/main/benchmark#result using the default params. This is certainly an open area of exploration, so any input or experiences you can share are welcome. Annotated samples also help. Newer detectors being developed as part of #482 may also provide higher accuracy across a wider range of inputs.

Also using PySceneDetect should the frame count start at 0 or 1? I have been using it starting it at 0 which makes sense to me but we are integrating with another older system which I am not sure exactly how all of that is handled. For example on one video it detects a new scene cut at a certain frame, which is all black, but then the very next frame is the actual good content that I want, but I could not seem to find the right detectors / settings to use to get that to be right and give me the cut at the start of the good content frame, not the all black frame right before it.

How are you outputting the frame counts? In general using 0 or 1-based indexing is supported by most API functions already. If you're using the scenedetect command, it could be specific to the output command/format. I believe CSV output via list-scenes uses 1-based indexing, but others such as save-qp use 0-based indexing.

[joshuabolick]

Thank you @Breakthrough for the response and sorry for the delayed response.

A big part of the project we are working on deals with older footage from VHS tapes, so really trying to find a very accurate way to detect the very first ad very end frame of static noise like old tv static, vhs rolling bars, rolling tracking, and vhs artifacts etc...

We tried so many different combinations of detectors and settings, I will have to dig through them which I will do, but also now I can attach a group of example images showing what we are trying to detect in a very accurate way (even if its slower etc)...

Please let me know what you think after looking at these attached images thanks again!

[Breakthrough]

Thanks for the response. Could you also generate a statsfile for some sample videos? It can be generated with the -s flag on the main command, e.g.:

scenedetect -i video.mp4 -s stats.csv detect-adaptive detect-threshold detect-hist

This will generate stats for each frame that can be used to tune thresholds. Could you provide a statsfile as well for a few videos? We could also consider handling the cases of (analog) static like this as it's own detector, since this does come up quite often in video preservation applications. If we're unable to come to an adequate solution please feel free to file a feature request for this.

It seems like we might need a hybrid of a threshold detection system and also a noise detector. We might need a golden file for this if we go that route, but it's feasible.

[joshuabolick]

Thanks @Breakthrough! Sorry for the delay but I am running this now for some videos and I will send over the results files a bit later today, thanks again!

[joshuabolick]

Also just to summarize for our use case, what we really need is the right combination of detectors and their settings:

• To frame accurate detect the very first and very end frame of any static noise like old tv static, vhs rolling bars, rolling tracking, and vhs artifacts etc...
• To frame accurate detect the very first and very end of any frames that are almost solid or solid of any color (white, gray, blue, black, any color really)
• To frame accurate detect the very first and very end of the frames of fade in and fade out transitions of any color (white, gray, black, etc)
• In essence really we want to frame accurate detect to the exact frame the start and end of any what we are calling "non-value" frames start and end that we are trying to isolate and be able to cut out vs the "value" content that we want to preserve etc...

Thank you, I will send over those results files just a bit later today for some examples...

[joshuabolick]

Here are 4 examples from 4 videos / test cases we have that hopefully should be good examples, I have not looked at the data yet but I can as well, but please let me know if this helps.

I can definitely find some more video examples to run this on to send some more examples too, I know its challenging because static and vhs artifacts are moving changing visuals but this would be extremely helpful if we can figure something out and please let me know any other questions or anything else I can do to help thanks again!

stats1.csv
stats3.csv
stats4.csv

[joshuabolick]

I have one more that looks like its 32MB so its too big to attach here it appears