Problem: I wrote a big long C program that printed output to stdout and also wrote some stuff directly to a log file, but the verbose output was also helpful sometimes. Naturally, I wanted to save that for later, so I did the usual thing to run the program:
./program > log.txt &
When I made my toy examples, included tons of diagnostic output, and ran it on smaller data sets, of course, things worked as expected. However, when I removed diagnostic output and ran it on larger images (ie, so it would take hours to run instead of minutes), suddenly, my logs were turning up empty even when I checked them hours later. I was worried! So I'd kill the job, tweak the file, recompile, and start doing more obscure things:
./program &> log.txt ./program 2>&1 | tee -a some_file
And even some options involving screen. The above two are probably your best options. Oddly, none of these worked. However, when I would run in the terminal, things would show up.
Solution: stdout, when it is not running to a terminal or terminal emulator, will buffer the output and release it in chunks. The buffer can be up to 4K. So, if your program is ouputing perhaps 10 lines over the course of 12 hours before closing, you might not get any output in your log if you do it this way until the program finishes. Therefore, either have the program write directly to a log file (if you want to glance at output along the way), have your program write a lot of output, or, in your program, use the following after printf commands where you really want to be able to view your results immediately-ish:
fflush(stdout);
Or whatever the equivalent command is outside of C. HTH. HAND.
In case you're wondering what I was up to, I was finishing up an incremental semi-supervised k-means algorithm. Exciting!
Now back to your regularly scheduled theological content.
No comments:
Post a Comment