Q:

Detect ending of frame in H.264 video stream

I am using JM reference software for encoding and decoding of video. What I want to do is to add Redundant bits using RS codes to the encoded video so that if some bits get lost during transmission we recover it on decoder side.
The problem I am getting is to detect end of frame when reading H.264 file.
So my questions are as follows

How to detect end of frame when reading h.264 file?
Also, how to detect end of GOP in h.264 stream?

A:

I think Technically speaking the only way to detect when a frame ends is to see where the next one begins. That means, you should search for the next start prefix code 0x000001. If you go to the previous data before this (which in the general case is the byte alignment syntax elements) and add your RS codes, a compliant decoder should ignore all that until the prefix code is seen.
As for the end of GOP, I'm afraid that the GOP concept does not exist in the Video standard, in theory you could start receiving frames which use previous frames as reference, but there is never a repeating structure. You can detect the end of a coded video sequence, but that does not necessarily coincide with what we know in the video community as "the GOP".
Edit:
In reply to the comment:
Lets suppose the bit stream is this (I am going to invent, it will be all garbage):
0x 00 00 01 e5 d8 31 04 ab 80 00 00 01 8e 56 04 94 cd 80 10 00 00 01 ...
You know for example that 00 00 01 e5 d8 31 04 ab 80 is the first frame, so you take all those bits and generate a CRC for them (or any other error correcting technique that you want), lets suppose you insert one byte of CRC ( that hypothetically will allow you to detect up to 5 bits in error and correct 3 for example), then your new bit stream would be
0x 00 00 01 e5 d8 31 04 ab 80 fc 00 00 01 8e 56 04 94 cd 80 53 10 00 00 01
here the CRC would be the bytes in boldface.
As for the GOP. Remember that even though when you decode you can see a repeating structure of frames "it is not required" to be like that forever. IF you want you could count the number of frames between I frames and in the practical cases it will work. Most H.264 encoders use a structure like I  {[B] (xN) P} (xM) I ...
Examples:
I B B P B B P B B I B B P B B P B B I B B P B B P B B ... etc
I B B B P B B B P B B B I B B B P B B B P B B B I B B B P B B B P B B B ... etc
But if you start counting and see in first example that the number of frames between two I frames is 8, nothing guarantees that in the next cycle it would be the same. Usually it will be, but there is nothing in the standard saying that you have to do it like that.
Hope it helps

