I currently have the following line of code.
(Get-Content \'file.txt\') | ForEach-Object {$_ -replace \'\"\', \'\'} | Set-Content \'file.txt\'
This should be faster than line-by-line processing, and still keep your memory consumption under control:
Get-content 'file.txt' -ReadCount 5000 | foreach-object {$_ -replace '"', '' | add-content 'newfile.txt' }