neumoDVB

After many years of work, neumoDVB is now close to ready.

NeumoDVB is a DVB Settop box and dx-program for linux.
Its supports blindscan and spectrum scan advanced muti-tuner cards such as the TBS-6909x and TBS-6903x,
but this requires installation of modified drivers, which can be found in one of my github repositories.

Except for blindscan and spectrum scan, neumoDVB should also work with the regular drivers and with
other DVB cards.

Some of the features include

• spectral analysis of satellite bands
• blindscan
• scanning muxes
• viewing and recording programs
• EPG scanning, including SkyUk, Freesat, Movistar, Viasat Nordic
• watching and recording encrypted streams using oscam
• tuning to multi-stream and T2MI streams (tested only 1 stream)
• DVB-S2, DVB-T (completely untested), DVB-C (almost untested)
• controlling positioners

Currently this software is fully functional, but is of alpha quality. It has seen
very little testing. Last week I started creating documentation and the most important
part of this documentation is ready. In the process of checking if what is written in the documentation
actually works, I also found and fixed several bugs.

The documentation is already available here:
GitHub - deeptho/neumodvb: neumoDVB DVB-S2/DVB-T/DVB-C settop box and DX program for Linux

In the next week or two, I will clean up te source code and then add it to the git repository.
The sofware will be released under the GPL V2 licence. It has been created in such a way that
creating ubuntu, debian, fedora... packages should not be too difficult.

In the mean time, comments are welcome.

Below are some screenshots.

Here is the channel list, after minimal configuration (channel numbers are still 0)



This is the EPG stram. In this case the EPG data originates from FreeSat (7 days EPG). The live channel is playing on the right.



Regular viewing is performed in full screen mode, but the window below has a similar layout. It also shows the on-screen display of signal levels
and at the bottom the control interface for time shift mode (currently limited to jumping forward/backward by 1 minute and pausing). The Penguin
keeps your children safe from any content you may watch.



Perhaps a gimmick, but neumoDVB allows simultenous viewing of multiple live channels. Video decoding uses the GPU when available and playing 4 to 8 live
channels might be feasible. Two is no problem. The yellow box indicates the active live channel, which is the one which can be retuned, time-shifted etc.



NeumoDVB's positioner control window, has the expected functionality to operate your rotor, but also allows tuning and blindscan. The example shows a multi-stream
on 5.0W.



The next screenshot shows a spectrum scan on top. Each of the spectral peaks can be individually blind-scanned, with the results shown at the bottom. It is
also possible to scan all muxes automatically. Old spectrums can also be loaded, for example to compare wit the current one.

Congratulations and thanks @deeptho on finishing your project I hope I can install it,but before that,I would like to know if the driver is restricted to a particular kernel or version of Ubuntu?
I now have Ubuntu 20.04 installed. The latest kernels of this create problems,not so much with installation,but because they change the resolution of the graphic card.
If it is an application for dx-ers,I would like to know if you noticed an improvement in blind scanning with cards 6903x/6909x compared to Windows?In other words, is it faster at scanning and how does it handle the weak signal?
And if you don't mind my asking,could you tell what the differences are between your version and,say,Chris Lee's (updatelee),apart from the fact that you adapted it for s2x cards?
I would ask too much if you could present a video of how to install,especially since some of us are not versed in Linux.
Thanks in advance

This application looks very interesting deeptho. I hope to have a better chance of success with this, as I failed to get your original neumoDVB work for me. (kernel issues).
You have provided some excellent documentation which will hopefully help me

Congratulations and thanks @deeptho on finishing your project I hope I can install it,but before that,I would like to know if the driver is restricted to a particular kernel or version of Ubuntu?

Click to expand...

Just in case it is not clear: the drivers are here:
GitHub - deeptho/linux_media: TBS linux open source drivers
They can be built with the well known media_build, but I placed a version that works here:
GitHub - deeptho/media_build: media_build for compiling media tree on this site

In principle it should work with any version, but once in a while changes in the generic linux kernel
causes incompatibilities with linux_media, which then fails to compile. Usually the problems are in other unrelated
drivers (which is what you experience).

I update the drivers from time to time by integrating changes from the official linux_media. It would of course be
better if my changes were integrated there or if they could be compiled "out of tree", but even then keeping up with
kernel changes is a continuous effort.

I am personally running kernel 5.10.19 on fedora 34. That kernel or any older one should work in theory.
Newer ones are sometimes a problem when some incompatible changes are made in the standard linux kernel.

What kernel version is your ubuntu using?

Perhaps you should just try compiling it and let me know if it works or not (things have changed since the initial version,
so problems may have gone or new ones may have grown).

In the longer term it would probably be best to support a few well chosen kernels, e.g., for the most used ubuntu and fedora versions
and then update them from time to time.

If it is an application for dx-ers,I would like to know if you noticed an improvement in blind scanning with cards 6903x/6909x compared to Windows?In other words, is it faster at scanning and how does it handle the weak signal?

Click to expand...

Well. my own experience on windows with these cards is quite poor. I have multiple hardware versions of them and

• ebspro, which is a pretty nice program does not work with them: no blindscan, no spectrum
• crazyscan, another nice program, works with some but not all of the hardware versions of the tbs6903x and tbs6909x but only on the first tuner. The reason is probably that the driver is made for the tbs6903x and does not take into account the extra tuners on tbs6909x. The spectrum is obviously wrong (spectrum shows artificial jumps), blindscan is sometimes working, but more often not.
• Some hardware versions ar enot working at all (probably because of an unoffficial hardware version number)
• No success with any other programs: crashes, or all kinds of problems

I have tried ebsPRO and crazyscan on other cards, with good results. I like ebsPRO best. In any case, I avoid windows when possible.

Of course the above trouble makes it impossible to compare speeds.
In general, about the speed of blindscan: you have to be careful when comparing:

• Some types of blindscan give less information, or miss more transponders.
• Some types of blindscan are faster than others on some satellites and slower on others
• Some satellites have weird or incompliant muxes which can cause slow downs or even some software to get stuck on them

So you will have to try for yourself. In any case, there is always a balance between quality of the scan and speed.
By the way: during development I scanned most satellites between 30.0W and 53E. On several of them weird problems occurred,
which required workarounds in the code. I even encountered it mux which claimed to be on 99.9 Ghz, for example. And then
there is the problem of spill-over: one one sat position receiving data from another sat. So this full arc scan was a good exercise
to track down (and fix) problems.

The fastest version of my blindscan code is in `neumo-blindscan`, which is already on github.
It does the bare minimum: finding muxes that can be locked ad discovering which multi-streams (if any) exist.
It tries to not miss any mux (and finds most, but with some exceptions).

It is pretty fast on satellites with strong signals (e.g., a few minutes in total), and slower on other satellites, because it wastes time on investigating
noise peaks. I could make it even 4 times faster by using tuners in parallel, but that is not implemented yet and would require
some more driver changes. I think in speed it probably compares to ebsPRO if ebsPRO would support this card (it does not).

The version in neumo-DVB uses the same approach as nuemo-blindscan, so will lock the same muxes.
However, it also checks SI data, which takes at least 30 seconds extra on each mux. So
I would consider it "slow and thorough". It could be made as fast as neumo-blindscan with 2 or 3 lines of code changes.
Personally I prefer to just use the spectrum and click on some trasnponders of interest. Within a few seconds I can see if
it locks, and if I wait a little more, I know what it contains.

And if you don't mind my asking,could you tell what the differences are between your version and,say,Chris Lee's (updatelee),apart from the fact that you adapted it for s2x cards?

Click to expand...

All code is completely different. I considered using his code, and studied it, but it did not really work on the cards I had at
the time, and I needed some incompatible changes. Also his kernel-tree was not compatible with some other drivers I needed (same
type of problem as you are experiencing). So I gave up, not because that code would have problems, but because I could not reuse it for the cards
I was really interested in anyway.

PS. Even now, I am not sure if that software supports scanning a full band, or only "blind tuning"
(i.e., clicking on the graph to tune it). If I remember correctly, this was one reason why I needed incompatible changes (support
full band blindscan)

So you will probably use updatelee or my code depending on the cards you want to use.
It is a pity that the two kernels are not compatible, but that is the way it is. It would not be too difficult
to have both approaches co-exist in the same kernel, but that requires some effort.

At some point, I also checked what changes in updateDVB would be needed to support my drivers, but I got stuck at some point for lack
of time, and never continued the search. So instead, I included the graphs ... in my own program.

I would ask too much if you could present a video of how to install,especially since some of us are not versed in Linux.

Click to expand...

The documentation has a whole section on installation and compilation. I would suggest starting from that. If something is
not clear, ask, and I will then update the documentation. As the software has never been run on Ubuntu, the first time Ubuntu-user
will probably need to make some extra effort, e.g., to find out exactly the names of the packages for prerequirsite software that neumo depends on. So if that first user takes some notes, I will add them to the documentation.

Making video is yet another effort. I might do it at some point, but isn't written version easier?

Llew said:

This application looks very interesting deeptho. I hope to have a better chance of success with this, as I failed to get your original neumoDVB work for me. (kernel issues).
You have provided some excellent documentation which will hopefully help me

Click to expand...

That was not neumoDVB, but neumo-blindscan.

neumo-DVB also runs on regular drivers, but requires the blindscan drivers for blindscan.

Did you try the latest version of the drivers? There have been some updates a few months ago.

Sorry, meant neumo-blindscan.

I haven't been back to trying neumo-blindscan for some time, so maybe updated drivers may have worked for me since then. I'll start afresh with your latest app., thanks deeptho.

deeptho said:

That was not neumoDVB, but neumo-blindscan.

neumo-DVB also runs on regular drivers, but requires the blindscan drivers for blindscan.

Did you try the latest version of the drivers? There have been some updates a few months ago.

Click to expand...

deeptho, I'm attempting to build the application on Fedora 34. Kernel 5.13.12.200. Neumo blindscan kernel drivers added.

The documented prerequisites added except mpv-libs-devel and libdvbcsa-devel, which weren't available. I found versions online and added to /usr/include. Is that OK?

Now, I'm now trying to build neumoDVB, but there's no CMake Lists.txt in neumodvb

Please advise

Llew said:

deeptho, I'm attempting to build the application on Fedora 34. Kernel 5.13.12.200. Neumo blindscan kernel drivers added.

The documented prerequisites added except mpv-libs-devel and libdvbcsa-devel, which weren't available. I found versions online and added to /usr/include. Is that OK?

Now, I'm now trying to build neumoDVB, but there's no CMake Lists.txt in neumodvb

Please advise

View attachment 138406

View attachment 138407

Click to expand...

There only seems to be Docs and Readme.md in the Github deeptho/neumodvb.

satesco said:

I now have Ubuntu 20.04 installed. The latest kernels of this create problems,not so much with installation,but because they change the resolution of the graphic card.

Click to expand...

Thanks a lot @CrazyCat
Yes, I managed to install the driver for kernel 5.11.0.-3 on Ubuntu 20.04 without changing the resolution,following the lines:
-wget https://www.tbsdtv.com/download/document/linux/media_build-2021-06-07-i915.tar.bz2
-tar xjvf media_build-2021-06-07-i915.tar.bz2
-cd media_build
-./install.sh
-reboot

Llew said:

There only seems to be Docs and Readme.md in the Github deeptho/neumodvb.

Click to expand...

Llew,
One of the links in the README.md points to the installation procedure:

Of course, it is better to wait until the code is actually there. It should not be long from now (I have been sick for three days with fever.
Fortunately not covid, but I could not do much).

In the mean time, I fixed some compilation issues which seem to occur with the newest version of clang (actually
a bug in clang) and some issues with DVB-C.

After neumoDVB is released I wil integrate crazycat's work in my drivers so that it also builds with the newest kernels
(my own kernel is 5.9.15-200, which is also fairly new).

I also have some ideas on how to make an out-of-kernel buld-tree with only the blindscan modules. The idea is that this could be
combined with any other source tree. This might be a more upgrade-proof solution for the long term. This for later, as it requires some
work.

deeptho said:

Llew,
One of the links in the README.md points to the installation procedure:

neumodvb/docs/INSTALL.md at master · deeptho/neumodvb

neumoDVB DVB-S2/DVB-T/DVB-C settop box and DX program for Linux - deeptho/neumodvb

github.com

Of course, it is better to wait until the code is actually there. It should not be long from now (I have been sick for three days with fever.
Fortunately not covid, but I could not do much).

In the mean time, I fixed some compilation issues which seem to occur with the newest version of clang (actually
a bug in clang) and some issues with DVB-C.

After neumoDVB is released I wil integrate crazycat's work in my drivers so that it also builds with the newest kernels
(my own kernel is 5.9.15-200, which is also fairly new).

I also have some ideas on how to make an out-of-kernel buld-tree with only the blindscan modules. The idea is that this could be
combined with any other source tree. This might be a more upgrade-proof solution for the long term. This for later, as it requires some
work.

Click to expand...

I should've checked the repo before starting a build. Apologies.

Meantime, I'll go back to trying to get a working neumo-blindscan on my Fedora distro - I had problems with it on Ubuntu.

Keep us informed on your progress with neumoDVB when you are better.

Llew said:

I should've checked the repo before starting a build. Apologies.

Meantime, I'll go back to trying to get a working neumo-blindscan on my Fedora distro - I had problems with it on Ubuntu.

Keep us informed on your progress with neumoDVB when you are better.

Click to expand...

Fever has gone...

On fedora, the best way to proceed is to first install a working fedora. I am currently on 34, but am using modules
compiled on an earlier version. With media_build you need a bit of luck.
It is important to install kernel-devel for the kernel you are running while building.

In case of troubles, you can use my modules, which will work on fedora 34 (and probaby any other
recent one like 31, or 33). The link below will autodestruct after a few weeks:
https://0x0.st/-wjc.gz
It should be installable like

cd /
sudo tar -xf /tmp/xxx.tar.gz
where /tmp/xxx.tar.gz is the file downloaded from the above link
This will install the modules (media_build + blindscan)


The next thing to do is generate an initramfs. On fedora this is highly automated, which is typically why things somtimes go wrong in mysterious ways. I use a command like
dracut -f /boot/initramfs-5.9.15-200.fc33.x86_64.img 5.9.15-200.fc33.x86_64
but you can try to official way of doing it, which involves /etc/dracut.conf

After that you need to update the grub config file. Again, this has become automated too much to my taste
but /etc/grub2.cfg is what you need.

Before doing any of this, save a copy of grub.cfg and related files (they could be in multiple places, but typically somewhere in /boot) Before rebooting double check the newly generated files by comparing them to the previous ones. Also keep the install disk at hand. With it, you can solve any boot-related problem, but it is not always easy.


PS. If you can find the exact kernel online (google for 5.9.15-200.fc33.x86_64 fc33 rpm) then maybe an easier approach is to install that official kernel first and get it working first. Afterwards you can then overwrite
the /lib/modules/5.9.15-200.fc33.x86_64/... files with the provides ones and everything should work (well
on modern computers with uefi may things can go wrong. Turn off secure boot in the bios if you cannot boot changed kernels....)

Of course none of this is needed if media_build works...

deeptho said:

Fever has gone...

On fedora, the best way to proceed is to first install a working fedora. I am currently on 34, but am using modules
compiled on an earlier version. With media_build you need a bit of luck.
It is important to install kernel-devel for the kernel you are running while building.

In case of troubles, you can use my modules, which will work on fedora 34 (and probaby any other
recent one like 31, or 33). The link below will autodestruct after a few weeks:
https://0x0.st/-wjc.gz
It should be installable like

cd /
sudo tar -xf /tmp/xxx.tar.gz
where /tmp/xxx.tar.gz is the file downloaded from the above link
This will install the modules (media_build + blindscan)


The next thing to do is generate an initramfs. On fedora this is highly automated, which is typically why things somtimes go wrong in mysterious ways. I use a command like
dracut -f /boot/initramfs-5.9.15-200.fc33.x86_64.img 5.9.15-200.fc33.x86_64
but you can try to official way of doing it, which involves /etc/dracut.conf

After that you need to update the grub config file. Again, this has become automated too much to my taste
but /etc/grub2.cfg is what you need.

Before doing any of this, save a copy of grub.cfg and related files (they could be in multiple places, but typically somewhere in /boot) Before rebooting double check the newly generated files by comparing them to the previous ones. Also keep the install disk at hand. With it, you can solve any boot-related problem, but it is not always easy.


PS. If you can find the exact kernel online (google for 5.9.15-200.fc33.x86_64 fc33 rpm) then maybe an easier approach is to install that official kernel first and get it working first. Afterwards you can then overwrite
the /lib/modules/5.9.15-200.fc33.x86_64/... files with the provides ones and everything should work (well
on modern computers with uefi may things can go wrong. Turn off secure boot in the bios if you cannot boot changed kernels....)

Of course none of this is needed if media_build works...

Click to expand...

I was just about to download your kernel version anyway, but just found this post.

I have Fedora 34.

I will go over the procedures you list and see what I come up with tomorrow. Thanks.

Would Mint be ok? I don't have the means to test this myself but I and others may wish to in the future.

I do not know Mint myself, but there is no reason why it would not work...

Alas, fell at the first hurdle, namely installing the -wjc zip.

Appeared to install (no errors for a change), but nowhere in the file system could I locate the mentioned media_build or blindscan directories.

Notice I'm using past tense - I've wiped Fedora from my PC, and will now go back to Ubuntu to try some earlier kernels: the version I have gives me the same problem.

I have released the neumoDVB software today.
Bugs should be reported using the issue tracker on github, and not here (this is not a technical forum; of course
other discussion is welcome here).

The code can be found here:
GitHub - deeptho/neumodvb: neumoDVB DVB-S2/DVB-T/DVB-C settop box and DX program for Linux

It code has seen very little testing, but most functions have been tested and were verified
to work at least once recently. To use it, you will need to be able to compile programs under linux.
As this software is new, you should expect some problems.

• blindscan functionality and spectrum scan only works with the appropriate DVB-cards and drivers (also on my github). Other functionality should work on any card with the standard drivers (except for bugs).
• DVBS(2)(X) is working and has been tested on many european sats. Multiistream is working (French/Italian on 5.0W).
• T2MI is "semi-working". That means: I have tested it only on "Sport Italia HD". It works with two problems: 1) picture distortion due to incomplete implementation of the specific video format by ffmpeg 2) no audio due to a bug in mpv. So in summary: neumoDVB code works, but some other problems exist. Even so, the stream can be saved to disk and then played (with picture distortion because most players are based in ffmpeg) using a standard video player. I would like to test on the Russian muxes on C-band, but I need a polarising plate for that and brexit problems have stopped me from receiving the one I ordered in the UK (which was expensive as well). Some small additional changes may be needed because of incomplete SI information on that satellite.
• DVB-T/T2 is unlikely to work out of the box. The main reason is that I have no possibility to test it locally (I can only receive one mux sporadically). However, all the code is there, and I expect only minor changes (probably in the tuning part. So the problems may be small but could still result in no picture).
• DVB-C works but of course I can only test on my own cable network
• Currently only standards in use in Western Europe are supported. Other standards shoulld not be too difficult to add, but that will depend on the help of a local user, preferably one with some compilation and debugging skills. Patches are welcome.
• encrypted streams work via oscam (this is meant to support local cards with a legal subscription - not illegal card sharing
• EPG is available from most satellite sources (jncluding freesat). xmltv is not supported (but not difficult to implement should someone want to invest time in it)
• Subtitles, audio track selection, timeshift, recording... functions are available. Some of the GUI aspects could/should be improved

Feature requests are welcome, but don't get your hopes up: my own time is limited.
Hopefully, some other developers will jump in at some point and the project will develop into a community supported project.

deeptho said:

Bugs should be reported using the issue tracker on github, and not here (this is not a technical forum; of course
other discussion is welcome here).

Click to expand...

If you mean technically advanced in Linux programming, fair enough, but always hoping there may well be someone on the (otherwise often quite technical) forum able to figure out some of the errors I occasionally find trying to get this working.
If this is a bug, I don't have the expertise to give my analysis on Github of the error I have. The log pointed to tells me nothing I can understand,, so I can't proceed further.
I have all the relevant prerequisite packages.
Clues, anyone?

BOOST was missing, or perhaps an unusable earlier version was in the package boost-program-options curl-devel.

Found a later version online.

Onwards and upwards...