This is a 2007 story. However to the best of my knowledge it is still generally relevant in 2020. Files continue to have headers that can get corrupted and can be fixed with "hex editors".
Publishing the Video "U. S. Aiono - What is Success?"
A case study in video file
disaster recovery.
As tapeless video shooting by writing files to a memory card
becomes more and more common, I hope this story of messing it up but then
recovering is useful to others. I found that recovery is possible but very
difficult and time-consuming. My NUMBER ONE advice is - if you see a
"change battery" warning then STOP.
I assumed that the camera would gracefully shut down saving
the file but that was NOT the case!
IN BRIEF
I was shooting an event with a camera recording video to a
memory card.
I was getting a low-battery warning but I kept going to try
to capture all of an almost-complete ceremony.
When the battery failed, the resulting file appeared on the
memory card
but displayed a size of "0 kilobytes".
I spent a lot of time downloading and trialling various
recovery software.
I found that it had limited capability to fix and I needed to do a lot of
manual "pre-fixing" to get close enough to the correct specs for the
fixing software to help.
My path to recovery was:
-
Run the Windows utility CHKDSK /F /R on the memory card.
This reported it had found "lost chains" and recovered the file
as "FILE0000.CHK".
I made backup and working copies of "FILE0000.CHK"
-
I discovered that "FILE0000.CHK" was the missing
video file but it had a blank header. Video Fixing software did not work.
I used the Cygnus Hex Editor Free Edition to copy and paste the header
from another video file made by the same camera.
-
With the new header, the file still would not play but the
error messages from a range of players looked more encouraging than
before.
I tried video fixing software again. Both "FixVideo" and
"AVI Previewer" were successful at identifying a "bad frame
1" and performing a fix.
The video file was now playable.
-
I found that "Windows Movie Maker" was
successful at identifying the correct length of the video. I used Windows
Movie Maker to export the file to the "DV" video standard to act
as the working file for editing.
IN DETAIL
The camera is a Canon MVX25i hybrid video and stills
camera. The MVX25i is the PAL version of a model better known in the NTSC
world as the "Canon Optura 40". It can record video to tape or to
memory card. The memory card video is of lesser quality but has
post-production convenience advantages.
The event was the 2007 Diploma Graduation ceremony for the Dept of Computer and
I.T. at the Manukau Institute of Technology. I was using memory card video:
(1) For quick and easy
publication of the result on a computer network
(2) Because I did not have
an external microphone or sound system patch available so I felt that I could
get better sound from the built-in mic without the whirring noises of the tape
drive motor.
I used up all of the following Saturday on the resulting
recovery, mainly because of lack of information causing me to follow a lot of
blind alleys. I persevered
(1) Because the guest
speaker Uluomatootua Aiono gave an excellent speech which I very much wanted to
save.
(2) Because investigating
the file recovery issue in order to write this article became an aim of the
exercise.
Step 1 - "CHKDSK"
This is a very old Microsoft Command-Line Utility that dates
from the days of DOS. I was unable to find it on my Windows Vista system at
the Institute so I had to copy everything and take it home to work with on a
Windows XP system.
CHKDSK is like an old friend which has in the past helped me
rescue files from failing hard drives. It was a relief to discover that it
worked just fine on a memory card and produced a recovery file.
Step 2 - Find a Header
I first tried renaming "FILE0000.CHK" to
"FILE0000.AVI" and playing it or fixing it in a lot of software
packages. All reported messages like "not a video".
After looking at the raw file in "Wordpad" I made
this guess:
Hypothesis: The Canon camera works by writing the file raw
with a blank header and on a Correct-Stop it then knows the file statistics( eg
how long it is) and can go back and fill in the header with that information.
Experiment: Copy the header from a Canon file of similar
size and Paste it into the blank space of this one. Then try to play it.
I tried with "Wordpad" and failed.
I tried with the binary editor in "Microsoft Visual
Studio".
I had 2 recovered video files from the card. "Visual
Studio" worked just fine with the smaller file of 126 Meg. But on loading
the important file of 267 Meg the copy/paste failed to work.
Off for another Internet search for downloads.
I found "
Cygnus Hex Editor FREE EDITION"
(Softcircuits, 2006).
This is simply excellent!
The layout, and the situation, is very clear from the
diagram.
FILE0000.CHK is on the left, and another avi file is on the
right.
The header from "LIST>" onwards matches
perfectly and the blank space is the correct size to match the avi file
header. Copying and pasting is straightforward.
Step 3
With the new header, the file still would not play but the
error messages from a range of players looked more encouraging than before.
I tried video fixing software again. The trial versions of both "FixVideo"
and "AVI Previewer" were
successful at identifying a "bad frame 1" and performing a fix. Note
that the trial of AVI Previewer is restricted to processing a video of max size
of 150 Meg so its success was with my second smaller recovered video.
"FixVideo" was the product actually used for recovering
"U. S. Aiono - What is Success?"
The video file was now playable.
I did a lot of experimentation with correcting the running
time.
The recovered video was 34 minutes long. But my borrowed
header was from a video of 7 minutes so the result at this stage showed up in "Windows
Media Player" and "ULead Mediastudio Pro 8 Video Editor" as a 7
minute video.
I was able to find out how to change that - details in the
appendix below - but that then turned out to be unnecessary because when I
later opened the video with "Windows Movie Maker" I found that it
ignored the borrowed header and assigned the correct length to the video.
Step 4
I used Windows Movie Maker to export the file to the
"DV" video standard to act as the working file for editing in ULead.
Appendix
How the number of frames is defined in AVI videos.
This is written as a DWORD, a 4 byte long integer, at Hex
position "30".
In the Canon files, I observed that this is written in 2
places. I edited them both the same. I did not have time to experiment with
leaving them different.
This is Hexadecimal numbering running from right to left
59 translates to 5 x16 + 9 = 89
21 translates to 2 x 16 + 1 = 33
The number of frames is therefore 33 x 256 + 89 = 8537
(Gruber, 2001)
Reference List
Aiono U., Calder J., Manukau Institute of Technology., 2007.,
"U. S. Aiono - What is Success?"
[Web Video].,
Independent Alternative Film-makers Channel on Youtube,
http://www.youtube.com/iafilm
Softcircuits, 2006. Cygnus Hex Editor FREE EDITION.
Softcircuits Website.,
Retrieved 02 Jun 2007 from http://www.softcircuits.com/cygnus/fe/
Video-Fixer.com, 2003. Video-Fixer. [Software
US$39.99], Video-Fixer.com Website.,
Retrieved 01 Jun 2007 from http://www.video-fixer.com
FDRLab, 2006. AVI Previewer.[Software US$19.99],
FDRLab Data Recovery Centre Website,
Retrieved 01 Jun 2007 from http://www.avi-previewer.com
Gruber, T., 2001., AVI Header Format.,
Fastgraph Website,
Retrieved 02 Jun 2007 from http://www.fastgraph.com/help/avi_header_format.html
[found via a Google search for "avi header"]