Some Shell Sed file treatments !
Some Shell Sed file treatments ! Preparation Lets connect to Manus server. ssh yug@[my ip] -p 822 passphrase : [press enter] password : [ask me!] cd ~/wikidata-scrapper/ tree Lets merges the kids into a Big Brother file: cat ./worker-00/zh_output_all.txt ./worker-01/zh_output_all.txt ./worker-02/zh_output_all.txt ./worker-03/zh_output_all.txt >> all.txt How to clean up my .txt ? Given lines more or less like : {Requests:288139, FR: got:50943, fail:237196, pc:0.17680008607} Lets delete lines with specific string marking empty french result : sed -i /fr=""/d all-test.txt or (not tested): A=fr="" sed -i "/$A/" all-test.txt My file thus get 20% of its initial size ! Then, delete all the meta data : sed -i s/{.*}// all.txt Its now ~12% of the initial size, or 39MB. Replace or delete the unwanted sections : #sed -i string type | target | replacement | global file.ext sed -i s|ar="[^"]*" cs="[^"]*"| |g all-test....