Remove Special Chars from Windows files

Jul 2017

If you have copied any text files or text strings from a Windows PC, then you might find that there are still special characters in this file that stop you executing it as a script on a linux computer. These special characters are typically carriage returns and line feeds.
Below is a file that I copied from Windows to a CentOS linux server, it dind't matter if I copied the file, or did a copy/paste, the issue still arose. After each line you can see a ^M character, which is a Windows carriage return (Linux doesn't use Carriage returns, only line feeds, Windows uses both).
When you edit the file in vi, you can't see these special characters, you have use another switch with vi to show them. Run vi using a -b switch.

vi -b filename

#!/bin/bash^M
# Install useful tools^M
^M
yum install epel-release -y^M
yum install htop -y^M
yum install nload -y^M
yum install iperf -y^M
yum install tcpdump -y^M
yum install ncdu -y^M
yum install net-tools -y^M
rpm –import http://packages.atrpms.net/RPM-GPG-KEY.atrpms^M
touch /etc/yum.repos.d/atrpms.repo^M
echo “[atrpms]^M
name=Fedora Core \$releasever - \$basearch - ATrpms^M
baseurl=http://dl.atrpms.net/el\$releasever-\$basearch/atrpms/stable^M
gpgkey=http://ATrpms.net/RPM-GPG-KEY.atrpms^M
enabled=1^M
gpgcheck=1” > /etc/yum.repos.d/atrpms.repo^M
yum install ffmpeg -y

You can remove these special characters in Linux using a variety of methods, here are two.

1. In vi, just delete the ^M characters and save the file
2. Install dos2unix (yum install dos2unix) and execute this on your file (dos2unix filename) and it will remove them for you, handy if it is a big file.