The gills will impart a bitter flavor to your fish, and will make it spoil faster, so they need to go. You can most easily do this by cutting them where they attach at either end of the arc that they form. I use kitchen shears, but you could use your knife. On small fish, you can just rip them out.

