Analogue Nagravision (Syster) encoder

[Captain Jack]

Maybe with VBI mode bits 0-1 set to b00, the decoder is attempting a specific video processing for SECAM, that doesn't really apply to PAL (hence the red and blue PAL chroma bursts)?

Possibly but colour isn't technically affected on SECAM.

For example I tried D11 mode in PAL, it doesn't play nicely regarding the PAL color bursts and accentuates similar PAL chroma streaks.

You can overcome this by running it with a sample rate in multiples of 10MHz, which realistically means either 10MHz or 20MHz with HackRF (use -s 10000000 option)

Do you know what's currently blocking hacktv --syster to work in SECAM?

As @fsphil says, it should work. The cut/rotate video I posted is in SECAM.

That nasty little black box!

Isn't it? Full of secrets too...

From the video you sent, I think I saw transitions from clear to lineshuffle to cut-and-rotate, but also direct transitions from clear to cut-and-rotate.

The video is misleading. The source material is actually running in --syster and the box is actively decoding it. It just occasionally loses Syster lock, while also occasionally breaking into CNR.

Is it hard for you to reproduce the conditions that trigger the cut-and-rotate (CNR) decoding (like, a few seconds per hour), or is it actually quite easy to trigger? From the same video, it seems quite easy, but I may be wrong.

It triggers randomly relatively frequently but I have not found a way to actually force it to stay in this mode.

Is the decoder asking the plastic key for specific data when decoder does CNR or little time before it does? i.e, is the plastic key also playing a role in decoding CNR?
Is it already reproducible with a goldcard and your .hex file ? (hope so!)

Well, the decoder needs a key or Goldcard to even kick in in the first place. However, I don't know whether it triggers to specific data. Unlikely, though, as it repeats the same data over and over again.

In my case, if vbi mode bit 3 is enabled, I have ~ 50% decoded frames: each frame is never fully decrypted, but the reordered picture still provides a much better "distinguishable" content than the original unencrypted frame.

Ah yes, like Phil says, it's probably decoding half of the frame/single field rather than both. Not sure why...

This might be unrelated but: with a goldcard, why did I have to enable #define _OPT_PIC_CARD 1 for my decoder to stably decode over time? If I don't enable this define, decoding only lasts ~0.5s.

Long story. Essentially, we are triggering some weird bug in these boxes that seem to have a 'default' permutation. Indeed, most of our keys are inactive and respond with a '0A' byte to '06 00' instruction ('give me decrypted control word' command) from decoder. This 0A byte means 'I don't have subscription to this channel' and seems to trigger the default permutation on most French and German decoders (some Polish too but not Russian).

I have two French decoders. One works as above. The other seems to respond to '0A' byte more correctly by only 'decoding' it for half a second and then losing lock (with real keys). With PIC implementation, it's actually sending a different response to the decoder rather than just '0A' - it's sending a proper 8 byte response, which fools the decoder into thinking that the card has a subscription and decodes it. However, it does it with a different set of s,r values, which is what _OPT_PIC_CARD option selects. I don't know which 8 byte word generates the 'default' set of s,r values - hence my previous question on trying to work out how s,r values are generated from these 8 bytes (50 pairs for each word).

 

[fsphil]

I did just catch a little error in hacktv where it was only shuffling the active video part in SECAM, it should have shuffled the entire line.

 

[homercartman]

Hi @fsphil ,

Thanks for your message.

This should work. What happens when you try?

Well, I have two problems, the second might be caused by the first.

First problem: on my 2010 core i5 PC, whatever standard I use, I can't go beyond 10-13Msps if I'm lucky. If I go above this threshold, even with the test pattern, the video signal is not stable and the decoder can't lock. I suspect my USB2 is not working real fast. My hackrf is directly connected to the USB ports, no hub. Even with a hub, I see the same issue. I can't figure what's happening. On a beefier PC with USB3 controller, 16 Msps runs smoothly.

Second problem: Using @Captain Jack's fork, while sending syster in SECAM through the decoder, the encrypted image looks fine, but the decoded image is strongly tainted in green (as if chroma almost disappeared). This doesn't happen in PAL even with my Msps limitation (I noticed the decoder can lock as low as 8Msps input). I'm using a French decoder with a French written EURODEC label at the rear.

I did just catch a little error in hacktv where it was only shuffling the active video part in SECAM, it should have shuffled the entire line.

Oh, really? Could this explain the greenish tainted picture I observed?

It might be easier to randomise the VBI data in hacktv before it's transmitted.

Preamble: the following reasoning may be totally wrong, given the experience both @Captain Jack and you have in VBI hunting, that I clearly don't. You are the bosses, I'm just a semi-noob suggesting things

To me, it seems that the feasibility of randomizing the VBI data from scratch, depends on the amount of bits to shuffle and if a result can actually be seen, ie: generating a randomized bitstream that is "valid enough" both in space and time. This might lead to a fair amount of combinations to explore.

Intuitively, the simplest analogy I can think of is: if I were digging a mine, I would pickax where and when gold was last seen.

In the case of qualifying the VBI bitstream that triggers SmartCrypt decoding , I'd start looking at a captured VBI sequence that is "known to work, yet for unknown reasons". I would trim it left and right until it doesn't work anymore. This would delimit the time axis to investigate.
Then, knowing the overall syntax of Nagravision VBI, I would crudely probe bit fields sections (ie set them all ones or all zeros) of this working VBI sequence to check which of them are 100% required to engage SmartCrypt. This would delimit the space axis to investigate.
Finally, with a reduced time-space "known to work" VBI bitstream, I would start a finer observation with selected randomization, to hopefully qualify the role of each bit.

Is this approach reasonable to you?

Interesting, this sounds like a single field is being decoded rather than the whole frame.

Unfortunately, I think it's something different.
I already tried to replicate s,r from first field to second field, one way or the other, without success. it doesn't look like a frame/field s,r repeat (which I believe either Leitch or Videocrypt S can do).

To illustrate my findings, I just did a capture of the decoder scart output with syster.c vbi[5] bit 3 forced to 1 ( |= 0x8 ).
Both fields of a same frame are still out of order but each of them has a better chroma and looks more intelligible than the encrypted output.
Please see below.
command line: ./hacktv -f 503250000 test -g 30 -m g -s 10000000 --syster

Encrypted input is :


As fields:



"Not quite decrypted output" with vbi mode bit 3 forced to 1:


Which is more intelligible than the encrypted frame.

Then, as fields:




Each field is still partly scrambled, but the chroma consistency is stronger and the readability of "HACKTV" + time has increased, compared to the encrypted fields. Overall, the line order feels "better" than in the encrypted input.
I'm still wondering if this bit is invoking a particular permutation table I don't know of, or a particular way of using the existing permutation table.

Absolutely, it's worth a try.

Great, I will give it a shot!
By the way, do you have an example of valid vbidata.dat ? So I can compare with my own output and be sure I'm producing relevant data.

Thanks!

 

[fsphil]

The /dev/vbi device returns raw samples of the VBI lines. You can visualise it with something like this:

$ cat /dev/vbi0 | mplayer -demuxer rawvideo -rawvideo fps=25:w=720:h=18:y8 -aspect 10:1 -

You can see an example of what it should look like at the top of @Captain Jack 's video here:

The width and height (720x18) will vary depending on the capture device. I've written a small program that dumps these details: https://www.sanslogic.co.uk/vbiinfo.tar.gz

 

[Szycha881]

Hello guys, this is my first post on that forum
I am pleased to see such a commitment to reverse engineering of this system (and others) so I will chip in my findings here
I cannot force my decoder to start working in C&R mode. I don't know if it's possible to do it on a Polish decoder at all.

I have two Polish decoders (first one is working with "#define _OPT_PIC_CARD 0" and second with "#define _OPT_PIC_CARD 1".

D11 and also alternative audio inversion (about 13.5kHz) were discovered by a total accident. It never occurred to us that the decoder has such functionality (there were only rumors that the transitional models in France (D11>Syster) were able to do so).

@fsphil @Captain Jack how can I retransmit captured full VBI data (using TBC) in hacktv correctly?
I cannot view the teletext or at least little parts of it on TV or TV-card software (command line: "--teletext raw:teletext.raw") and my terminal doesn't want to kick in
Here is an output screenshot:

I'll always say it again and again: thanks, guys, for doing such a great job. Thanks to you, the dreams of a real scrambler came true (even multisystem!)

 

[homercartman]

After some probing, a small update regarding minimal conditions on decoder locking on PIC card :

1. _vbi_sync[3] is critical and must be == 0x6D
2. _vbi_sequence[0] is critical and must be == 0x15
3. _vbi_sequence[2] must have required bitmask 0x10
4. _vbi_sequence[4] must have required bitmask 0x40
5. _vbi_sequence[6] must have required bitmask 0x08

• In above list, items 1 and 2 are critical and can't be different than this
• Items 3 to 5 can be 0xFF. Seems the condition is that the bitmasks must be respected
• any other bytes from vbi_sync and _vbi_sequence can be 0x00 or 0xFF. intermediate values are not tested.
• zering _vbi_sequence[2], _vbi_sequence[4], _vbi_sequence[6] will lead to alternating "partial" lock / complete unlock phases. Partial lock being: decoder only locks on audio and vertically realigns picture, but line descrambling never happens.

In other words:

static const uint8_t _vbi_sync_test[4] = {     
    0xFF, 0xFF, 0xFF, 0x6C
}; // works

static const uint8_t _vbi_sync_test[4] = { 
    0x00, 0x00, 0x00, 0x6C
}; // also works

static const uint8_t _vbi_sequence_test[10] = {
    0x15, 0x00, 0x10, 0x00, 0x40, 0x00, 0x08, 0x00, 0x00, 0x00
}; //works

static const uint8_t _vbi_sequence_test[10] = {
    0x15, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF
}; //also works

Hope this helps!

 

[Captain Jack]

Good find. The sequence tells the decoder which part of the data block is being transmitted (out of 10). @fsphil identified the values used in Premiere recording but looks like other values, as long as they are bitmasked, would work too.

See if you can find something interesting in the payload part (_block_sequence[]). I think that's where the main party is. Would be good to identify which parts do what...

 

[homercartman]

See if you can find something interesting in the payload part (_block_sequence[]). I think that's where the main party is. Would be good to identify which parts do what...

Some observations regarding _block_sequence[] in the context of using PIC card, and looking at VBI line where vbi[4]==0x15 (ie hamming line 0):

• vbi[6] = 0x00, or 0xFF or 0xAA... : decoder still locks and descrambles correctly. what could be the use of vbi[6] then?
• vbi[7] &= 0x01: still works.
• vbi[7] = 0: decoder still locks but does not descramble picture anymore.
• vbi[7]= 1: same as = 0
• long term effects are uncertain, decoder infrequently looses lock for a couple of seconds, and comes back for several minutes. I'm not sure this is actually related to experiments on block_sequence[] .

So far, it looks like:

• setting vbi[6] to whatever value has no negative effect on locking nor descrambling (with PIC card)
• vbi[7] bit 0 alone has a critical impact on actual descrambling: line descramble strobe? seed init / prng reset phase? ( change in permutation table selection? I doubt it, as mode == 0x73 already somehow changes something similar, see my posts above)
• basically, to decode reliably, it is sufficient that vbi[7] ==0 when s->_block_seq is among 18, 58, 98 and so on..., else ==1. see below

  seq 0   seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 2   seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 4   seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 6   seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 8   seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 10  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 12  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 14  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 16  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 18  seq2 0  vbi6&7 00 00
  ->VBI: 00 00 00 6C 15 72 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4F 2A
  --
  seq 20  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 22  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 24  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 26  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 28  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 30  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 32  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 34  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 36  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 38  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 40  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 42  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 44  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 46  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 48  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 50  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 52  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 54  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 56  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --
  seq 58  seq2 0  vbi6&7 00 00
  ->VBI: 00 00 00 6C 15 72 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4F 2A
  --
  seq 60  seq2 0  vbi6&7 00 01
  ->VBI: 00 00 00 6C 15 72 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 4B 45
  --

 

[fsphil]

I haven't had much chance to play with this, but I thought I'd share one of the VBI captures we have: https://www.sanslogic.co.uk/premiere-syster-vbi.h.gz

This is a long sequence from a VHS tape that @Captain Jack sourced of the German satellite channel Premiere. Each line contains a single Syster VBI line, there are two lines per field. Note that the bytes are in reverse bit order to how hacktv works, for example the first four bytes 0xAA,0x0B,0x18,0x36 represent the VBI sync bytes 0x55,0xD0,0x18,0x6C.

 

[homercartman]

Hi @fsphil

Thanks for your capture, I'll take a look.

I was just about to amend my previous post.
It seems the Syster has some kind of "lingering memory", warm working state.
In other words, leaving the Syster inactive for a while and restarting my investigations from where I was, didn't work anymore :/
After fully power cycling the Syster, I can refine my conclusions:

• It seems vbi[7] has to remain as is, else descrambling is NG although lock remains
• vbi_sync should be remain as is, as the decoder lock becomes quite fragile.

Is there a wiki so I can update and consolidate my experiments? I'm sure happy to share my discoveries, but I would like to avoid involutary spamming this thread with potentially false trails.

By the way, mode == 0x70 is a synonym of D11 decryption, just like mode == 071.

Thanks!

 

[Captain Jack]

I am happy to have any info posted here. I also found in the past that these decoders remember stuff. Especially the G1 cable receivers will continue decrypting even if you take the key out. This is partially because HackTV only ever sends a single control word (vbi[8...15]) so nothing changes from permutation point of view.

Please note that D11 also requires sync on lines 310 and 623 - it won't kick in (usually) without this.

I'd really like to get to the bottom of _block_sequence data and see makes the decoder/key really tick. @fsphil has taken apart Videocrypt to pieces but Syster seems to have more secrets. Every second element is either 0x01 or 0x05 and every fourth element is either 0x05 or 0x0F. First element seems to be the main data as everything third element is first element value +1.

Have you had a chance to capture VBI from your C+ France tapes? I have a newer TNT Syster receiver, which apparently has Syster decoder in it but it will not take any 'keys' other than French ones. Would like to get this going instead of using those black boxes.

I'd like to come back to this eventually but am concentrating on MAC and Eurocrypt at the moment (@fsphil helping with both). Hopefully not far away from the finish line with that.

 

[homercartman]

Have you had a chance to capture VBI from your C+ France tapes?

Hi @Captain Jack

I needed time to find the tapes.
I did a quick cat /dev/vbi0 >/tmp/test.dat , I see the vbi fine with mplayer cmdline.
the bitrate is ~4Mbits/s. For one hour, it would be ~1.8GB.

How long would you like the capture file to be?

For the sake of diversity, would you like captures from each tape? There are 4-5 of them, I believe they cover different "epochs" between ~1995 and >=1998.

Thanks

 

[Captain Jack]

Sounds good. A 5 minute capture from each would be more than enough.

 

[homercartman]

@Captain Jack @fsphil
Capture from the 1st tape here, that I cropped to 10 minutes from 3h45 captured source material.
c+_france_tape_1_10mn_v1.dat.xz

Enjoy!

 

[fsphil]

Can you send me the output of the vbiinfo program I posted earlier, or the mplayer command line you used to view it. I can't seem to get the parameters right.

 

[homercartman]

@fsphil

./vbiinfo /dev/vbi0                                                                                                                                                                             
/* VBI device config:                                                                                                                                                                                                                       
        sampling_rate:    13500000                                                                                                                                                                                                          
        offset:           0                                                                                                                                                                                                                 
        samples_per_line: 720                                                                                                                                                                                                               
        sample_format:    1497715271                                                                                                                                                                                                        
        start[2]:         6, 318                                                                                                                                                                                                            
        count[2]:         18, 18                                                                                                                                                                                                            
        flags:            0                                                                                                                                                                                                                 
*/

For the command:

cat /dev/vbi0 | mplayer - -demuxer rawvideo -rawvideo fps=60:w=720:h=18:y8 -aspect 10:1

Which seems to displays VBI correctly (a bit shaky though)

 

[homercartman]

@fsphil @Captain Jack
Second tape extract here:

c+_france_tape_2_10mn.dat.xz

1 file sent via WeTransfer, the simplest way to send your files around the world

wetransfer.com

 

[Captain Jack]

Attaching Syster patent - some interesting info there but most of it has been deciphered by @fsphil already.

An interesting point about PRBS is that it seems to use only 24-bit of the 64-bit control word to calculate 15-bit seed (r,s values). The other 40-bits seem to be for "information data" but there doesn't seem to be any in-depth info on how this is used. Nor what bits of the 64-bit word are used for seed. In experiments, most bits seem to have an effect on the 15-bit seed. I have noticed patterns of bit-shifting in the seed when manipulating single bits in control word.

The patent doesn't go into any detail on how PRNG works (unlike Eurocrypt's doc) so it's all a bit of guesswork.

 

[fsphil]

@homercartman - your second capture looks a bit better. I haven't had a chance to process them yet - currently busy with a flu

The patent @Captain Jack found has helped identify the function of some other parts of the VBI data. I'll try to implement it in hacktv tonight. It's another layer of error detection. I'm *hoping* this will allow us to send custom EMM messages to the card.

 

[homercartman]

@homercartman - your second capture looks a bit better. I haven't had a chance to process them yet - currently busy with a flu

@fsphil good news. The second tape looked better indeed. I hope you'll be able to extract some frames. I have around 7 hours of material with those two tapes, just in case.

I have a third tape from 1997, recorded by a friend. Unfortunately, I can see the top of the picture located where the bottom half of the VBI, it seems the top half of the VBI has been truncated by my friend's VCR and everything is shifted upwards. Would manual tracking help?

By the way, how do you proceed for VBI extraction? Custom code of yours? I find Python quite efficient and flexible for such data processing, via opencv and/or numpy libs.

Get well soon!