I was writing thousands of lines to a .csv file, but there were a lot of duplicate lines. So the only feasible way i could think of to fix it is to store each line in a char array, then before i actually write a line out, write the line to a temp char variable, and check that var against every line stored in the char array. (or the DB, if you will) If the line isn't in the array, it is a distinct one, so it can be added to the array, and written to the file.
Little did i know, it would actually work. So now i'm down to only writing out about 100 lines, which is the right amount for this particular file. All is well, and I like C coding.