Corrupted video files? Missing data - randomly CHANGING missing data?

[BlueRidgeMark]

At least if they're going to be corrupted, they could be corrupted consistently! Not so!

I have had two or three files in my Sage Recordings folder that are downright weird. I can see the files and the sizes seem to be what I'd expect, but the files will not play properly. I have tried playing them in Sage, Windows Media Player, and importing them into TMPG DVD Author 2.0. One of the files is so corrupted that Media Player and TMPG will not even open it. Sage will play about a minute of it, looping around to play that same minute over and over again.

Another file will open without any problem (any player), but it seems that some information has been lost about 10-20 minutes into the recording. This is a continuous recording of several shows. As I was authoring the DVD, I noticed that the first show was apparently about 36 minutes long. Hmmm. Too long. Going back over it carefully, I noticed that one person instantly changed outfits! (Same set.) Obviously, there were two shows merged together, but the position of the merge changed every time I watched it. At one point, I was able to see the opening frame of the second show about 10 minutes into the recording; another time it was about 18 minutes in. When I viewed the audio graph along with it, I found that the audio did NOT match the video. Show 1's audio continued while Show 2's video played.

Other recordings have been fine so far.

Has anyone else run into a similar problem? Can anyone suggest a cure? I am using an IBM NetVista P4 1.9 GHz with 640 meg of RAM. My capture card is the Hauppauge MCE 500, fed by composite video cables from an STB, and I'm using version 6 of SageTV under Windows 2K. HD is a new Maxtor 300 gig, PATA. It’s a slave drive, used only for capture. OS and programs are on the primary drive.

It occurred to me that one possible reason for the problem was that I tried to watch each of these recordings while the show was being recorded. I know that this shouldn't be a problem, but...

[BlueRidgeMark]

Well, we have another one. Any ideas?

[Opus4]

Since there haven't been any comments so far, turn on debug logging and contact support. Keep sample recording files.

Have you run any hard drive diagnostics?

Since you mentioned the problem occurring when you watched at the same time it was recording, is the recording drive formatted using 64K cluster size? If not, that could easily cause problems.

Those are my only guesses at the moment.

- Andy

[flavius]

One of my drives for recording is a 'slow' ATA100 drive with a cluster size of 4k, all my sata drives all 4k drives as well, and I never defrag (not in the last four years).

Never seen any corruption. Can you put the recording drive on another master?

[Opus4]

Originally Posted by flavius

One of my drives for recording is a 'slow' ATA100 drive with a cluster size of 4k, all my sata drives all 4k drives as well, and I never defrag (not in the last four years).

You seem to have been lucky. It won't necessarily give you corruption & it won't always stutter if the recording rate is low enough, but if you start getting stuttering, be sure to try changing the cluster size to 64K.

It can become impossible for the drive's head to seek from cluster to cluster fast enough to keep up with the video data throughput requirements.

- Andy

[flavius]

Originally Posted by Opus4

You seem to have been lucky. It won't necessarily give you corruption & it won't always stutter if the recording rate is low enough, but if you start getting stuttering, be sure to try changing the cluster size to 64K.

Hm, for the past four years I have been recording almost everything in DVD Standard (~3.2 gig/hr). Seems to be low enough.

I never (ever) had any stuttering, I had 0MB recordings with BTV galore, none with Sage - and one really weird recording of a House show in slow-mo.

For a client-server environment the cluster size should be irrelevant.

[Opus4]

Originally Posted by flavius

For a client-server environment the cluster size should be irrelevant.

Client-server doesn't have anything to do with it -- I'm only talking about the drive(s) where recordings are stored on a Windows system (I don't know about other OSes), and am not referring to any OS drive or the drives on the client where there are no recordings.

If you go to a command prompt, run chkdsk on your recording drives, and it reports that they all use "4,096 bytes in each allocation unit.", then all I can say is I'm glad it is working for you. I certainly won't say that you will definitely have problems at some point, but increasing the cluster/allocation size has helped a lot of people & it is one thing that should be checked.

- Andy

[flavius]

Originally Posted by Opus4

Client-server doesn't have anything to do with it..

I have a question for you: What do we have when there is no client-server setup?

.. then all I can say is I'm glad it is working for you.

You are welcome.

I certainly won't say that you will definitely have problems at some point, but increasing the cluster/allocation size has helped a lot of people & it is one thing that should be checked.
- Andy

We want a system with communicating pipes that doesn't leak - aka the appliance

The HD extender, well - if it ever arrives may actually qualify.

[Opus4]

Originally Posted by flavius

I have a question for you: What do we have when there is no client-server setup?

You have a stand-alone SageTV installation w/no clients. But, I don't understand your question in terms of the topic about cluster size -- you said "For a client-server environment the cluster size should be irrelevant." Whether you are in a client-server environment or not is irrelevant to the topic of cluster size. Whether you are playing on a client or playing on the server, cluster size is only important in terms of reading/writing from/to the drive fast enough to keep up with the stream of video data. You could have all your clients turned off, but the server still has to write the recordings to hard drives and read one recording at the same time if it is playing one.

Bringing clients or network storage into the discussion means adding network throughput considerations.

I only asked the original poster about cluster size on his recording disk & will leave that as my question to him for the time being. If he doesn't know, he can run chkdsk to see what allocation unit size it reports.

Remember: watching live TV means recording it to a file & reading that file for playback.

- Andy

[BlueRidgeMark]

Originally Posted by Opus4

Since there haven't been any comments so far, turn on debug logging and contact support. Keep sample recording files.

Will do!

Originally Posted by Opus4

Have you run any hard drive diagnostics?

No, it's a brand new drive. I know - that's no guarantee! I will do so at the first opportunity!

Originally Posted by Opus4

Since you mentioned the problem occurring when you watched at the same time it was recording, is the recording drive formatted using 64K cluster size? If not, that could easily cause problems.

I'm ashamed to say I dunno. But I will find out, and make sure it gets that way.

I'll get the debug logging turned on, do the diagnostics, and take care of the cluster size, then run for a couple of weeks, or until I get another corrupted file, whichever comes first. THEN I'll bug support. (No sense taking up their time unless I have something for them to work on.)

If I don't have any problems in a couple of weeks, I'll post back.


Thanks for the help, Andy!

[BlueRidgeMark]

Hmmmm. Maybe it's not cluster size after all. Take a look at this thread.

On my end, I cannot find a way to format for 64k clusters. Whether I use Windows tools or the manufacturer's software (Maxtor's MaxBlast), the only options I have are for 4096 bytes or smaller clusters.

Any ideas?

[rickgillyon]

Command Prompt:

format v: /FS:NTFS /A:64K

where v: is the drive letter.

[BlueRidgeMark]

Thanks, Rick. I will give that a shot. Have to wait now until my next break in the recording schedule, on Tuesday. Whether this problem is cluster size related or not, I do want to go to 64k, just to be on the safe side.

[icmoney]

Originally Posted by BlueRidgeMark

Well, we have another one. Any ideas?

No ideas, but I had a wierd occurance last week. I recorded two American Idol shows(in OTA HD) and when I went to watch them, every time I tried to skip or FF the show would begin over from the beginning. I finally let them play and they played all the way to the end without incident. All my other recordings before and after the American Idol recordings played normally with Skip and FF allowed. These two recording were the only ones I had on Fox and I have no idea which of my two HDTV tuners were used. I'm going to let it record again this week to see what happens before I dig into it.

[marsh9man]

Originally Posted by icmoney

No ideas, but I had a wierd occurance last week. I recorded two American Idol shows(in OTA HD) and when I went to watch them, every time I tried to skip or FF the show would begin over from the beginning. I finally let them play and they played all the way to the end without incident. All my other recordings before and after the American Idol recordings played normally with Skip and FF allowed. These two recording were the only ones I had on Fox and I have no idea which of my two HDTV tuners were used. I'm going to let it record again this week to see what happens before I dig into it.

I was having the same exact issue, and I finally figured out what my problem was...my harddrive needed to be defragged.
Maybe this will work for you.

[BlueRidgeMark]

No joy. I reformatted to 64k clusters on Tuesday, and Friday we got a corrupted file. It will play in Windows Media Player, but freezes at random points. What I mean is, play it the first time, and the image may freeze at 14 minutes, play it another time and it may freeze at 12 minutes, the next time at 18 minutes, and so on. The time keeps ticking, so it's not a hung program. Watch it long enough, and it unfreezes. Same in SageTV.

Using TMPG Author 3, the file will suddenly show thumbnails that do not match the main preview. Again, WHERE in the file this happens varies from run to run.

The actual file SIZE seems to be what we'd expect, as it is almost identical to the same capture done the week before. However, the TIME is wrong. There should be 1 hour 15 minutes of video. It IS 1:15 when viewed on our DVR (which is capturing the same video source in parallel), so we know the problem is NOT the video source.

When the Sage file is viewed in any media player or editor on our computer, it shows about 12 minutes LESS video. Because the actual file size looks correct, and the "missing" frames seem to change from one run to the next, it doesn't seem to be actually missing any video data. Rather, it's almost like it is confused about how to decode it. A sync issue of some kind?

Any ideas? I've read of people doing this successfully on much less powerful computers, so I have a hard time believing that's the problem. A 1.9 Gig P-4 with 640 meg of RAM should be no problem.

In fact, the only weird thing about our hardware is that the capture drive is slave on the secondary IDE channel (DVD drive is master), rather than being slave on the primary. THis is because the IBM Netvista puts its primary HD in a weird location, using a single connector cable. I'd need a custom IDE cable to connect mine to the primary IDE channel. A standard cable just won't reach.

[rickgillyon]

Download videoredo from videoredo.com (free trial).
Take a copy of your file and use videoredo to do a Quickstream Fix on it. At the end vrd will tell you what it had to fix, post that here.

Try the resulting file, chances are it will play without issues, you are probably just getting sync errors on the stream.

[BlueRidgeMark]

Thanks, Rick. I'll give that a shot.

[BlueRidgeMark]

VideoRedo is able to fix the files, even some that would crash anything that tried to open them. It's reporting sync errors. I've enabled logging, but can't find a log file! Nothing in the help file, either. Is the "log" just the screen that pops up and reports the errors at the finish?

Anyway, that's a useful diagnostic and a nice band-aid, and thanks very much for the tip, Rick. It has saved a LOT of lost time for us. Our recording isn't for entertainment - it's a satellite download of our children's education. Video classes. Losing recordings is not trivial!

However, it doesn't fix the underlying problem. It's beginning to look like this really is a horsepower problem. File corruption seems to happen when trying to do other things (like video editing or DVD authoring) on the computer while it's recording. We are keeping track of that to see if we can confirm that, but based on memory, that seems to be the case.

If so, it doesn't make much sense, as other people have reported no problem with similar configurations.

[kpsmith]

I've had a similar problem when ripping or authoring DVDs while recording 1 or 2 shows. Bumping memory up to 1 GB resolved the issue for me.

I really didn't start seeing the issue until i upgraded to 2 tuners and then it just seems like it was too much for the machine at the end of the day.