Mounting NTFS filesystem in Linux

NTFS or New Technology File System is Microsoft's proprietary filesystem used in Windows NT, Windows 2000 and Windows XP. It is a robust filesystem having many more features like encryption, robust security, support for quotas and so on which the FAT(32) filesystem lacks. If you are dual booting your machine between windows NT/2000/XP and Linux, and if you have a NTFS partition on your hard disk, then at some point of time, you might have a need to access the data on the NTFS partition from Linux. But since NTFS is proprietary, it is next to impossible to fully support it from Linux. Which means, there is no way to write data to an NTFS partition from Linux at this point of time. The good news is you can mount the NTFS volume as read only and access your data from Linux. Here I will explain how you can achieve this.

First gather some data about your kernel. Find out the type of kernel you are using.

# uname -r
2.6.5-1.358

This is needed because you need to install a module providing NTFS support to your kernel and you have to make sure the module you download have the same version number as your kernel for this to work.

Now visit linux-ntfs.sourceforge.net . I am using Fedora Core 2. So I will be explaining it with respect to Fedora. If you have a different RedHat version, then you have to visit the section on the website related to your version of Redhat. And if you are using any other distribution like debian, you can build your own module from the source provided on the website.

Now I downloaded the NTFS kernel module kernel-ntfs-2.6.5-1.358.i686.rpm from the website and installed it - logging in as root.

# rpm -ivh kernel-ntfs-2.6.5-1.358.i686.rpm
Preparing... ############################### [100%]
1:kernel-ntfs ############################### [100%]

There should be no errors, just #'s. If all goes well, you will get the message "The Linux NTFS rpm has been successfully installed".

Now execute the following command to see whether linux recognizes NTFS partition on your harddisk:

# fdisk -l /dev/hda

This gave the output on my machine as follows:

Disk /dev/hda: 10.2 GB, 10262568960 bytes
255 heads, 63 sectors/track, 1247 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 510 4096543+ 7 HPFS/NTFS <---o
/dev/hda2 511 523 104422+ 83 Linux
/dev/hda3 524 548 200812+ 82 Linux swap
/dev/hda4 549 1247 5614717+ f W95 Ext'd (LBA)
/dev/hda5 549 1122 4610623+ 8e Linux LVM

As you can see above, Linux recognizes the /dev/hda1 NTFS partition as HPFS/NTFS, which is what we want.

The last step is mounting your NTFS partition in linux. This is achieved by using the mount command as follows:

# mkdir /mnt/c_drive
# mount -t ntfs -r -o umask=0222 /dev/hda1 /mnt/c_drive

Also if you want to mount the NTFS partition automatically each time you boot Linux, append the following line in your /etc/fstab file.

#FILE: /etc/fstab
/dev/hda1 /mnt/c_drive ntfs ro,defaults,umask=0222 0 0

Now you are done. If you want to know more about NTFS, then read this very good NTFS FAQ .

Compiling a linux kernel from source

Something I found really wonderful in linux is that you can compile a custom kernel for your machine including just the features that are needed. You might think why anyone would want to compile a custom kernel from source when you get a precompiled binary kernel, right? For one, suppose you have an old PC on which you want to run linux; And you know that you have no need for USB and PCMCIA support because you don't have eaither on your PC. Won't it be really nice to recompile your kernel from source without support for these ? Because then you will be reducing the size of the kernel and optimizing the kernel for better performance.

Ofcourse with the most recent hardware, there is no need for recompiling because the performance gains are not noticeable. But you can get better performance if you run a custom kernel on old hardware with limited amount of memory.

I had earlier writen about the steps needed to compile linux kernel 2.4.x . But the steps needed to compile the latest 2.6.x version kernel is a lot different from 2.4.x versions.I came across an interesting article with details about patching a kernel to compiling both 2.4.x and 2.6.x kernels from source. You can read all about it at Digital Hermit.

The /proc filesystem

Linux has become really popular as a server operating system mainly due to its security and stability. Infact the system administrators could do system maintanence tasks short of any hardware upgradations without rebooting the machine. That means there is virtually no downtime suffered by a linux server. This is made possible because Linux provides various ways to change the underlying operating system values and settings while keeping the system up and running. Linux contains a virtual filesystem called /proc which can be accessed by the system administrators to achieve the above tasks.

A /proc filesystem is not a real filesystem because it resides only in the computer's memory and does not utilize any space on the hard disk. This filesystem is a map to the running kernel process. The /proc filesystem is mounted in the /proc directory during system initialization and has an entry in the /etc/fstab file.

If you move into the /proc directory, you will find a lot of sub directories and files. Some of these files give information on system hardware, networking settings and activity, memory usage and so on of your computer. And there are some other files in the /proc/sys directory whose values you can manipulate to make changes to the various parameters of the running kernel settings.

Here I will describe the various files and directories residing in /proc that a system administrator could find helpful.

If you list the files under /proc, you will find that all the files have a size of zero - this is because they are not really files and directories in the typical sense. If you want to view the contents of a file in the /proc directory, you use the 'cat' command.

WARNING: Do not use 'cat' on /proc/kcore as this is a special file which is an image of the running kernel's memory at that particular moment - cat'ing this file will leave your terminal unusable.

Some of the key files in the top-level directory are as follows :

• /proc/interrupts - View IRQ settings
• /proc/cpuinfo - Information about the system's CPU(s)
• /proc/dma - Direct Memory Access (DMA) settings
• /proc/ioports - I/O settings.
• /proc/meminfo - Information on available memory, free memory, swap, cache memory and buffers. You can also get the same information using the utilities free and vmstat.
• /proc/loadavg - System load average
• /proc/uptime - system uptime and idle time. Can also be obtained using utility uptime.
• /proc/version - Linux kernel version, build host, build date etc. Can also be obtained by executing `uname -a`.
  

Beneath the top-level /proc directory are a number of important subdirectories containing files with useful information. These include :

• /proc/scsi - Gives information about SCSI devices
• /proc/ide - information about IDE devices
• /proc/net - information about network activity and configuration
• /proc/sys - Kernel configuration parameters. The values in files in this directory are editable by root, which I will further explain below.
• /proc/ - information about process PID.
  

/proc/sys directory

As explained earlier, this directory holds most kernel configuration parameters and is the one that is designed to be changed while the system is running. Some of the files which are of real use to the system administrators are as follows:

• /proc/sys/fs/file-max
  
  This specifies the maximum number of file handles that can be allocated. If some of your users get an error when trying to open more files stating that the maximum limit of number of open files have been reached, then you need increase the value in this file (default is 4096) to set the problem straight as follows :
  

  # echo "10000" > /proc/sys/fs/file-max

• /proc/sys/fs/super-max
  This specifies the maximum number of super block handlers. Any filesystem you mount needs to use a super block, so you could possibly run out if you mount a lot of filesystems.
  Default setting: 256
• /proc/sys/kernel/acct
  

  This holds three configurable values that control when process accounting takes place based on the amount of free space (as a percentage) on the filesystem that contains the log:
  

  1. If free space goes below this percentage value then process accounting stops.
  2. If free space goes above this percentage value then process accounting starts.
  3. The frequency (in seconds) at which the other two values will be checked.

  To change a value in this file you should echo a space separated list of numbers.
  Default setting: 2 4 30
  

  These values will stop accounting if there is less than 2 percent free space on the filesystem that contains the log and starts it again if there is 4 or more percent free space. Checks are made every 30 seconds.
• /proc/sys/kernel/ctrl-alt-del
  This file holds a binary value that controls how the system reacts when it receives the ctrl+alt+delete key combination. The two values represent:
  

  1. A zero (0) value means the ctrl+alt+delete is trapped and sent to the init program. This will allow the system to have a graceful shutdown and restart, as if you typed the shutdown command.
  2. A one (1) value means the ctrl+alt+delete is not trapped and no clean shutdown will be performed, as if you just turned the power off.

  Default setting: 0
• /proc/sys/kernel/domainname
  

  This allows you to configure your network domain name. This has no default value and may or may not already be set.
• /proc/sys/kernel/hostname
  This allows you to configure your network host name. This has no default value and may or may not already be set.
• /proc/sys/net/ipv4/ip_forward
  

  This allows you to turn on/off IP forwarding. If the value in this file is "1" then ip forwarding is turned "on "and if value is "0" then ip forwarding is turned off.
  

This is only a small subset of 100's of configuration parameters that can be changed via the files in /proc/sys directory.

But the /proc/sys modifications are temporary and are not saved at system shutdown (which may be a rare instance as far as servers are concerned). But you can use the "sysctl" command to manage such settings in a static and centralized fashion. It reads the values in the /etc/sysctl.conf file. The sysctl command is called during boot time by the /etc/rc.d/rc.sysinit script. So to make the changes you make in the kernel parameters permanent, just enter it in the /etc/sysctl.conf file and then execute the command :

# sysctl -p

... to make the kernel reread the changes from the /etc/sysctl.conf file.
There are two simple rules for converting between files in /proc/sys and variables in sysctl:

• Drop the /proc/sys from the beginning.
• Swap slashes for dots in the filenames.
  

These two rules will let you swap any file name in /proc/sys for any variable name in sysctl.
So /proc/sys/net/ipv4/ip_forward will become net.ipv4.ip_forward
There is an interesting article about the /proc filesystem at Linux Gazette.

Resizing Logical Volumes

This is a continuation of my earlier post Creating Logical Volumes in Linux . Here I will explain how to resize an existing logical volume. Logical volumes may be resized dynamically while preserving the data on the volume. Here is how:

Reducing a logical volume

1. Reduce the filesystem residing on the logical volume.
2. Reduce the logical volume.
   

For different file systems, it is achieved differently.

For ext2 file system

If you are using LVM 1, then both the above steps could be acomplished by executing a single utility called e2fsadm.

# umount /data
# e2fsadm -L -1G /dev/my_vol_grp/my_logical_vol
# mount /data

The above command first reduces the filesystem in the 'my_logical_vol' by 1 GB and then reduces the my_logical_vol itself by the same amount.

If you are using LVM 2 - more recent linux distributions like Fedora use LVM 2 - then you do not have the 'e2fsadm' utility. So you have to first reduce the filesystem using 'resize2fs' and then reduce the logical volume using 'lvreduce'.

# umount /data
# resize2fs  /dev/my_vol_grp/my_logical_vol  1G
# lvreduce  -L  1G  /dev/my_vol_grp/my_logical_vol
# mount /data

In the above case, I have reduced my file system "to" 1 GB size ...

Note: I didn't use the minus (-) sign while using resize2fs

... And then used the lvreduce command to reduce the logical volume "to" 1 GB. If I want to reduce the logical volume "by" 1 GB, then I give the same command but with "-L -1G" instead of "-L 1G".

Reiserfs file system

If you have a reiserfs filesystem, then the commands are a bit different than ext2(3).

# umount /data
# resize_reiserfs -s -1G /dev/my_vol_grp/my_logical_vol
# lvreduce -L -1G /dev/my_vol_grp/my_logical_vol
# mount -t reiserfs /data

XFS and JFS filesystems

As of now, there is no way to shrink these filesystems residing on logical volumes.

Grow a Logical Volume

The steps for growing a logical volume are the exact opposite of those for shrinking the logical volume.

1. Enlarge the logical volume first.
2. Then resize the filesystem to the new size of your logical volume.
   

Update (July 22nd 2005) : I came across this very interesting article on LVM at RedHat Magazine which I found really informative.

Creating a LVM in Linux

I am sure anybody who have used windows (2000 and above) have come across the term dynamic disks. Linux/Unix also have its own dynamic disk management called LVM.

What is an LVM ?

LVM stands for Logical Disk Manager which is the fundamental way to manage UNIX/Linux storage systems in a scalable manner. An LVM abstracts disk devices into pools of storage space called Volume Groups. These volume groups are in turn subdivided into virtual disks called Logical Volumes. The logical volumes may be used just like regular disks with filesystem created on them and mounted in the Unix/Linux filesystem tree. The logical volumes can span multiple disks. Even though a lot of companies have implemented their own LVM's for *nixes, the one created by Open Software Foundation (OSF) was integrated into many Unix systems which serves as a base for the Linux implementation of LVM.

Note: Sun Solaris ships with LVM from Veritas which is substantially different from the OSF implementation.

Benefits of Logical Volume Management

• LVM created in conjunction with RAID can provide fault tolerance coupled with scalability and easy disk management.
• Create a logical volume and filesystem which spans multiple disks.
  
  By creating virtual pools of space, an administrator can create dozens of small filesystems for different projects and add space to them as needed without (much) disruption. When a project ends, he can remove the space and put it back into the pool of free space.
  

Note : Before you move to implement LVM's in linux, make sure your kernel is 2.4 and above. Or else you will have to recompile your kernel from source to include support for LVM.

LVM Creation
To create a LVM, we follow a three step process.

Step One : We need to select the physical storage resources that are going to be used for LVM. Typically, these are standard partitions but can also be Linux software RAID volumes that we've created. In LVM terminology, these storage resources are called "physical volumes" (eg: /dev/hda1, /dev/hda2 ... etc).

Our first step in setting up LVM involves properly initializing these partitions so that they can be recognized by the LVM system. This involves setting the correct partition type (usually using the fdisk command, and entering the type of partition as 'Linux LVM' - 0x8e ) if we're adding a physical partition; and then running the pvcreate command.

# pvcreate /dev/hda1 /dev/hda2 /dev/hda3
# pvscan

The above step creates a physical volume from 3 partitions which I want to initialize for inclusion in a volume group.

Step Two : Creating a volume group. You can think of a volume group as a pool of storage that consists of one or more physical volumes. While LVM is running, we can add physical volumes to the volume group or even remove them.

First initialize the /etc/lvmtab and /etc/lvmtab.d files by running the following command:

# vgscan

Now you can create a volume group and assign one or more physical volumes to the volume group.

# vgcreate my_vol_grp /dev/hda1 /dev/hda2

Behind the scenes, the LVM system allocates storage in equal-sized "chunks", called extents. We can specify the particular extent size to use at volume group creation time. The size of an extent defaults to 4Mb, which is perfect for most uses.You can use the -s flag to change the size of the extent. The extent affects the minimum size of changes which can be made to a logical volume in the volume group, and the maximum size of logical and physical volumes in the volume group. A logical volume can contain at most 65534 extents, so the default extent size (4 MB) limits the volume to about 256 GB; a size of 1 TB would require extents of atleast 16 MB. So to accomodate a 1 TB size, the above command can be rewriten as :

# vgcreate -s 16M my_vol_grp /dev/hda1 /dev/hda2

You can check the result of your work at this stage by entering the command:

# vgdisplay

This command displays the total physical extends in a volume group, size of each extent, the allocated size and so on.

Step Three : This step involves the creation of one or more "logical volumes" using our volume group storage pool. The logical volumes are created from volume groups, and may have arbitary names. The size of the new volume may be requested in either extents (-l switch) or in KB, MB, GB or TB ( -L switch) rounding up to whole extents.

# lvcreate -l 50 -n my_logical_vol my_vol_grp

The above command allocates 50 extents of space in my_vol_grp to the newly created my_logical_vol. The -n switch specifies the name of the logical volume we are creating.

Now you can check if you got the desired results by using the command :

# lvdisplay

which shows the information of your newly created logical volume.

Once a logical volume is created, we can go ahead and put a filesystem on it, mount it, and start using the volume to store our files. For creating a filesystem, we do the following:

# mke2fs -j /dev/my_vol_grp/my_logical_vol

The -j signifies journaling support for the ext3 filesystem we are creating.
Mount the newly created file system :

# mount /dev/my_vol_grp/my_logical_vol /data

Also do not forget to append the corresponding line in the /etc/fstab file:

#File: /etc/fstab
/dev/my_vol_grp/my_logical_vol /data ext3 defaults 0 0

Now you can start using the newly created logical volume accessable at /data mount point.
Next : Resizing Logical Volumes

The Process of Chip making explained

What is the use of an OS (be it windows or linux or any other) if you do not have the right hardware to install it on ? Excluding MacOS, which runs on PowerPC architecture, almost all linux and windows Oses run on PC's that use either Intel, IBM or AMD processors. Have you wondered what these processors are made up of and how they make such powerful processors ?

These powerful processors are made on silicon wafers. Experts say that AMD, IBM, Intel and other heavy hitters all employ the same principles when it comes to manufacturing devices on a silicon wafer. Major differences in terms of the production process arise due to the type and number of process steps and the use of tools and materials. Did you know that AMD's latest processor Athlon AMD-64 contains 105.9 million transistors?

I came across an interesting article at Tom's Hardware which gives the exclusive inside story on AMD's chip production intersperced with lots of pictures of their manufacturing facility.
Really Interesting !!

Prevent a non-root user from shutting down or rebooting the system

To prevent all non-root users from using the shutdown, reboot or halt commands, do the following :

1. In the file /etc/X11/gdm/gdm.conf , change the line that reads :
   

   SystemMenu=true

   to
   

   SystemMenu=false

2. In the file /etc/inittab, change the line that reads :
   

   ca:ctrlaltdel:/sbin/shutdown -t3 -r now

   to
   

   ca:ctrlaltdel:echo "You are not authorized to turn off the machine"

3. In the directory /etc/security/console.apps/, delete the file reboot, poweroff and halt.
4. Remove the file /usr/bin/poweroff

Now only the root user will be able to turn off or reboot the machine.