android - Video rendering is broken MediaCodec H.264 stream -

i implementing decoder using mediacodec java api decoding live h.264 remote stream. receiving h.264 encoded data native layer using callback (void onrecvencodeddata(byte[] encodeddata)), decode , render on surface of textureview. implementation completed (retrieving encoded streams using callback, decode , rendering etc). here decoder class:

public class mediacodecdecoder extends thread implements myframeavailablelistener {      private static final boolean verbose = true;     private static final string log_tag = mediacodecdecoder.class.getsimplename();     private static final string video_format = "video/avc"; // h.264     private static final long mtimeoutus = 10000l;      private mediacodec mmediacodec;     surface msurface;     volatile boolean m_bconfigured;     volatile boolean m_brunning;     long startms;      public mediacodecdecoder() {         jniwrapper.setframeavailablelistener(this);     }      // callback receiving encoded streams native layer      @override     public void onrecvencodeddata(byte[] encodeddata) {         if(!m_bconfigured && bkeyframe(encodeddata)) {             configure(msurface, 240, 320, encodeddata);         }         if(m_bconfigured) {             decodedata(encodeddata);         }     }      public void setsurface(surface surface) {         if (msurface == null) {             msurface = surface;         }     }      public void start() {         if(m_brunning)             return;         m_brunning = true;         start();     }      public void stop() {         if(!m_brunning)             return;         m_brunning = false;         mmediacodec.stop();         mmediacodec.release();     }      private void configure(surface surface, int width, int height, byte[] csd0) {         if (m_bconfigured) {             log.e(log_tag, "decoder configured");             return;         }         if (msurface == null) {             log.d(log_tag, "surface not available/set yet.");             return;         }         mediaformat format = mediaformat.createvideoformat(video_format, width, height);         format.setbytebuffer("csd-0", bytebuffer.wrap(csd0));         try {             mmediacodec = mediacodec.createdecoderbytype(video_format);         } catch (ioexception e) {             log.d(log_tag, "failed create codec: " + e.getmessage());         }          startms = system.currenttimemillis();         mmediacodec.configure(format, surface, null, 0);         if (verbose) log.d(log_tag, "decoder configured.");          mmediacodec.start();         log.d(log_tag, "decoder initialized.");          m_bconfigured = true;     }      @suppresswarnings("deprecation")     private void decodedata(byte[] data) {         if (!m_bconfigured) {             log.e(log_tag, "decoder not configured yet.");             return;         }         int inindex = mmediacodec.dequeueinputbuffer(mtimeoutus);         if (inindex >= 0) {             bytebuffer buffer;             if (build.version.sdk_int < build.version_codes.lollipop) {                 buffer = mmediacodec.getinputbuffers()[inindex];                 buffer.clear();             } else {                 buffer = mmediacodec.getinputbuffer(inindex);             }             if (buffer != null) {                 buffer.put(data);                 long presentationtimeus = system.currenttimemillis() - startms;                 mmediacodec.queueinputbuffer(inindex, 0, data.length, presentationtimeus, 0);             }         }     }      private static boolean bkeyframe(byte[] framedata) {         return ( ( (framedata[4] & 0xff) & 0x0f) == 0x07);     }      @override     public void run() {         try {             mediacodec.bufferinfo info = new mediacodec.bufferinfo();             while(m_brunning) {                 if(m_bconfigured) {                     int outindex = mmediacodec.dequeueoutputbuffer(info, mtimeoutus);                     if(outindex >= 0) {                         mmediacodec.releaseoutputbuffer(outindex, true);                     }                 } else {                     try {                         thread.sleep(10);                     } catch (interruptedexception ignore) {                     }                 }             }         } {             stop();         }     } } 

now problem - streams being decoded , rendered on surface video not clear. seems frames broken , scene distorted/dirty. movement broken , square shaped fragments everywhere (i sorry don't have screenshot right now).

about streams - h.264 encoded , consists of frames , p frames (there no b frame). every frame has sps + pps + payload structure. color format used during encoding (using ffmpeg in native layer) yuv420 planner. sent length of data native layer okay (width * height * (3 / 2)).

during configure() set csd-0 value sps frame. frame used configuration frame (sps + pps + payload) - prefix sps frame, think configuration successful. note that, didn't set csd-1 value pps frame (is problem?).

every frame has preceding start codes (0x00 0x00 0x00 0x01) both p-frame , i-frame (for i-frame start code present both infront of sps , pps frame).

moreover, setting presentation timestamp system.currrenttimemillis() - starttime every frame increasing order every new frame. think shouldn't cause problem (correct me if wrong).

my device nexus 5 google android version 4.4.4 , chipset qualcomm msm8974 snapdragon 800. using surface decoding, think there should not device specific color format mismatch issues.

i can provide textureview code if needed.

what might cause of incorrect decoding/rendering? in advance!

edit 1

i tried manually passing codec-specific data(sps , pps bytes) during configuration. didn't make change :(

byte[] sps  = {0x00, 0x00, 0x00, 0x01, 0x67, 0x4d, 0x40, 0x0c, (byte) 0xda, 0x0f, 0x0a, 0x68, 0x40, 0x00, 0x00, 0x03, 0x00, 0x40, 0x00, 0x00, 0x07, (byte) 0xa3, (byte) 0xc5, 0x0a, (byte) 0xa8}; format.setbytebuffer("csd-0", bytebuffer.wrap(sps));  byte[] pps = {0x00, 0x00, 0x00, 0x01, 0x68, (byte) 0xef, 0x04, (byte) 0xf2, 0x00, 0x00}; format.setbytebuffer("csd-1", bytebuffer.wrap(pps)); 

i tried trimming start codes (0x00, 0x00, 0x00, 0x01) no progress!

edit 2

i tried hardware accelerated {{textureview}} mentioned in official documentation (though didn't find h/w acceleration code in sample project of mediacodec-textureview). still no progress. commented h/w acceleration code snippet.

edit 3

the screenshots avilable now:

edit 4

for further clarification, h.264 encoded i-frame hex stream format:

00 00 00 01 67 4d 40 0c da 0f 0a 68 40 00 00 03 00 40 00 00 07 a3 c5 0a a8 00 00 00 01 68 ef 04 f2 00 00 01 06 05 ff ff 69 dc 45 e9 bd e6 d9 48 b7 96 2c d8 20 d9 23 ee ef 78 32 36 34 20 2d 20 63 6f 72 65 20 31 34 36 20 2d 20 48 2e 32 36 34 2f 4d 50 45 47 2d 34 20 41 56 43 20 63 6f 64 65 63 20 2d 20 43 6f 70 79 6c 65 66 74 20 32 30 30 33 2d 32 30 31 35 20 2d 20 68 74 74 70 3a 2f 2f 77 77 77 2e 76 69 64 65 6f 6c 61 6e 2e 6f 72 67 2f 78 32 36 34 2e 68 74 6d 6c 20 2d 20 6f 70 74 69 6f 6e 73 3a 20 63 61 62 61 63 3d 31 20 72 65 66 3d 31 20 64 65 62 6c 6f 63 6b 3d 31 3a 30 3a 30 20 61 6e 61 6c 79 73 65 3d 30 78 31 3a 30 78 31 20 6d 65 3d 68 65 78 20 73 75 62 6d 65 3d 30 20 70 73 79 3d 31 20 70 73 79 5f 72 64 3d 31 2e 30 30 3a 30 2e 30 30 20 6d 69 78 65 64 5f 72 65 66 3d 30 20 6d 65 5f 72 61 6e 67 65 3d 31 36 20 63 68 72 6f 6d 61 5f 6d 65 3d 31 20 74 72 65 6c 6c 69 73 3d 30 20 38 78 38 64 63 74

and p-frame:

00 00 00 01 41 9a 26 22 df 76 4b b2 ef cf 57 ac 5b b6 3b 68 b9 87 b2 71 a5 9b 61 3c 93 47 bc 79 c5 ab 0f 87 34 f6 40 6a cd 80 03 b1 a2 c2 4e 08 13 cd 4e 3c 62 3e 44 0a e8 97 80 ec 81 3f 31 7c f1 29 f1 43 a0 c0 a9 0a 74 62 c7 62 74 da c3 94 f5 19 23 ff 4b 9c c1 69 55 54 2f 62 f0 5e 64 7f 18 3f 58 73 af 93 6e 92 06 fd 9f a1 1a 80 cf 86 71 24 7d f7 56 2c c1 57 cf ba 05 17 77 18 f1 8b 3c 33 40 18 30 1f b0 19 23 44 ec 91 c4 bd 80 65 4a 46 b3 1e 53 5d 6d a3 f0 b5 50 3a 93 ba 81 71 f3 09 98 41 43 ba 5f a1 0d 41 a3 7b c3 fd eb 15 89 75 66 a9 ee 3a 9c 1b c1 aa f8 58 10 88 0c 79 77 ff 7d 15 28 eb 12 a7 1b 76 36 aa 84 e1 3e 63 cf a9 a3 cf 4a 2d c2 33 18 91 30 f7 3c 9c 56 f5 4c 12 6c 4b 12 1f c5 ec 5a 98 8c 12 75 eb fd 98 a4 fb 7f 80 5d 28 f9 ef 43 a4 0a ca 25 75 19 6b f7 14 7b 76 af e9 8f 7d 79 fa 9d 9a 63 de 1f fa 6c 65 ba 5f 9d b0 b0 f4 71 cb e2 ea d6 dc c6 55 98 1b cd 55 d9 eb 9c 75 fc 9d ec

i pretty sure stream's correctness rendered using ffmpeg decoding , glsurfaceview opengles 2.0.

i took h.264 dump both native layer , java layer , found dump of native layer played java layer's dump played broken decoded stream. problem - during passing encoded streams native layer java, encoded stream not passed properly(corrupted) , because of buggy implementation (sorry following thread inconvenience).

moreover passing i-frame's payload decoder resulted in broken rendering. passing complete nal unit (sps + pps + payload) , okay :)