Links2Linux Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.

====== Latin1 to Utf-8 ======

===== Tools/Services which must be checked =====
  *  NFS
  *  Samba
  *  Windows
  *  sbclient
  *  rsync
  *  musync

===== Config changes =====
  *  Samba - smb.conf
unix charset = ISO8859-1 -> unix charset = UTF-8 (not tested yet)

===== Converting the filenames =====
  *  convmv
Just test what would be done:
<code>
 convmv -r -f latin1 -t utf-8 /path/to/files</code>

===== Converting textfiles =====
  *  recode
To convert a textfile that contains latin1 characters to utf-8 format you can use the recode tool
<code>
 recode latin1..utf-8 <filename></code>
converts a file from latin1 to utf-8 format

A problem can be if you happen to  convert a file which is already in utf-8 format.

I wrote the follwowing script that will convert only files with an iso-8859* charset to utf-8. Each file will be backed up before converting.

convert_files.sh:
<code>
  - !/bin/sh
  - 
  -  Convert latin1 files to utf-8 files
  - 

BAK_EXT=".convert_backup"

while read FILE; do
<code>
  if file -i "$FILE" | cut -d":" -f2 | tr -d ' ' | grep "^text/" | cut -d"=" -f2 | grep -q "iso-8859"; then
    if echo "$FILE" | grep -q "$BAK_EXT\$"; then
      echo "Skipping $FILE"
      continue;
    fi
    echo "Converting $FILE"
    cp -a "$FILE" "$FILE"$BAK_EXT
    recode latin1..utf-8 "$FILE"
  fi</code>
done
</code>

The script will read filenames from stdin. To convert all textfiles in your hone directory use
<code>
 find ~ -type f | convert_files.sh</code>


--[[user:mschiff|mschiff]] 13:25, 19 Apr 2005 (CEST)

Links2Linux Wiki

User Tools

Site Tools

Page Tools