Converting videos can be a painful process, especially if you take the brute-force approach of re-encoding the data. Not only does it take time, but the quality will suffer. If you just need to wrap the audio/video data inside a different container, FFmpeg can usually perform the transformation in seconds.
FFmpeg is command-line media manipulation tool you can download for free from the following website. Windows users will want to grab the all-in-one executable and place it in the same directory as the videos you want to convert (or, if you're feeling adventurous, you can edit the PATH environment variable so its accessible anywhere).
Then, fire up the command line, navigate to the folder in question and enter:
ffmpeg -i "[input]" -flags +global_header -vcodec copy -acodec copy "[output]"
[output] with the source video filename and the file to store to converted data. All this does is copy the video and audio streams into a different container format -- no recompression is done and the quality remains unmodified.
For instance, if you used:
ffmpeg -i "game.avi" -flags +global_header -vcodec copy -acodec copy "game.mp4"
Then the file "game.avi" would be converted to an identical MP4 of the same name.
Why might you want to do this? It's well know that non-linear video editing tools such as Adobe Premiere and Sony Vegas don't play well with certain container / codec combinations. Premiere for instance seems to work best with MOV files containing H.264 video and AAC audio.
You might hit a few speed bumps when converting AVIs to other formats. Take AVI -> MP4. If the AVI has an audio stream that is uncompressed, FFmpeg will throw out an error. This is because MP4 can only have audio in a few specific formats: MP3 and AAC being the most popular ones.
This means you will have to re-encode the audio for this conversion, which can be handled like so:
ffmpeg -i "[input]" -flags +global_header -vcodec copy -strict -2 -acodec aac -b:a 256k "[output]"
This tells FFmpeg to use its in-built AAC encoder to compress the audio stream before storing it in the output file. FFmpeg's AAC encoder isn't the best available, but at 256kB/s, the quality will be more than good enough. If you want to use something better, you'll have to compile FFmpeg yourself with certain flags set or just use the LAME MP3 encoder (though this might be less compatible with NLEs).
FFmpeg [Official site]