AIX and USB memory sticks

Creating file systems with USB memory sticks

USB memory sticks are cheap, cheap, cheap. As long as your hardware has USB slots, using these memory sticks is an option to consider for files that require fast reads, large data copies, or temporary file storage. This article demonstrates how to mount a USB stick on your AIX® box and use it for storage.

Share:

David Tansley (david.tansley@btinternet.com), System Administrator, Ace Europe

David TansleyDavid Tansley is a freelance writer. He has 15 years of experience as a UNIX administrator, using AIX the last eight years. He enjoys playing badminton, then relaxing watching Formula 1, but nothing beats riding and touring on his GSA motorbike with his wife.


developerWorks Contributing author
        level

01 November 2011

Also available in Chinese Russian

Note from the editor: There is limited USB support in AIX and only specific devices are supported. Other devices might work, but IBM does not support their usage. JFS2 file systems are not officially supported on USB devices, but you can try this at your own discretion.

Tasks that involve text file data extraction, where performance is of the essence and the text files are to be placed on temporary storage, requires some thought. Typically, one could consider using a RAM disk (file system), a USB memory stick, or perhaps on a more permanent basis using SSD (solid state disk). RAM disk file systems are extremely fast, but at some point you need to move the files from the file system since you could lose everything if the file system is unmounted or the system is rebooted. Plus, you need to have enough memory (pinned) to create the RAM disk in the first place. For SSD, this would be the ideal solution, but these are not as cheap as the USB memory devices (when the requirement is temporary moveable storage). Another consideration is that you cannot place an SSD or RAM disk in your pocket and re-plug it into another machine like you can with a USB (memory) stick. These sticks are also known as:

  • pen drives;
  • memory sticks;
  • USB sticks;
  • flash drives; and
  • thumb drives.

In this article, I will demonstrate how to create a USB memory stick file system (hereafter called USB stick) and compare the different speeds on text extraction using a 2MB CSV file. I am using AIX 7.1 with a 4GB Sandisk memory stick. These USB sticks can come in sizes up to 256GB.

Memory sticks are useful in my opinion because:

  • They are ideal for fast data access.
  • They can be used in data transfer where network performance is poor.
  • They are great for scratch or temporary file systems.
  • They are cheaper than other alternatives.

The only downside is that they do have a limited life in that the write cycles on these drives typically last around 8-10 years, before the memory cells wear out. But as they are cheap to buy, this is really not a concern as you would no doubt replace them within that time period anyway.

The AIX 6.1 documentation states that it supports the following branded USB sticks:

  • Sandisk
  • Lenovo
  • Kingston

However, I have used other branded and non-branded USB sticks with no issues found.

USB sticks are ideal for creating scratch file systems where you want to transfer a large amount of data, via courier, for quick extraction to a remote destination sie cheaply because there is no good network link. As well, you could use them for data manipulation of large text files. USB sticks are great for fast reads but does not offer good performance when writing to them; you need to keep in mind what type of processing task you use it for.

Mounting a USB stick

Be sure to have the correct fileset(s) installed before inserting your USB stick. The following file-sets are required to access a USB stick:

# lslpp -L devices.usbif.08025002.rte
 Fileset Level State Type Description (Uninstaller)
devices.usbif.08025002.rte
7.1.0.15 A F USB Mass Storage Device
 Software 
# lslpp -L devices.common.IBM.usb.rte
  Fileset                      Level  State  Type  Description (Uninstaller)
  devices.common.IBM.usb.rte
                            7.1.0.15    A     F    USB System Software

Insert the USB stick and run cfgmgr to discover it:

# cfgmgr

Once discovered, you have two devices (the block and raw device):

ls -l *usbms0
cr--r--r--    1 root     system       44,  0 Aug 22 18:59 rusbms0
br--r--r--    1 root     system       44,  0 Aug 22 18:59 usbms0

It is also shown in the devices output:

# lsdev |grep usb
usb0       Available               USB System Software
usbhc0     Available 08-08         USB Host Controller (33103500)
usbhc1     Available 08-09         USB Host Controller (33103500)
usbhc2     Available 08-0a         USB Enhanced Host Controller (3310e000)
usbms0     Available 2.1           USB Mass Storage

For AIX 5.3, the first USB device is shown as flashdrive0.

At this point, I could treat the USB stick as a raw volume and write data to it. However, in this demonstration, I will mount the USB stick as a file system. I feel there is no need to create a log device for this USB mount, since the purpose of a scratch file system is that it is temporary and thus speed is of the essence when transferring data. To create the USB stick file system use the following command:

# mkfs -V jfs2 -o ea=v2 /dev/usbms0
mkfs: destroy /dev/usbms0 (yes)?
File system created successfully.
3927736 kilobytes total disk space.
Device /dev/usbms0:
  Standard empty filesystem
  Size:           7855472 512-byte (DEVBLKSIZE) blocks

The previous command initialized the USB stick. Select "Yes" to destroy, or rather initialize, the device. The file system type is of jfs2 type. I do not believe that a type of jfs2 really matters for this task, as the file system is not going to use the jfs2 log device. But as this is the norm when creating file systems, I created it as a jfs2. To ensure the file system is scalable, I specify that with the 'ea' option. In this example, the device to use is the USB stick inserted, which is usbms0.

For AIX 5.3, there is no NOLOG option in the mount command.

Next, mount the file system specifying that is it to be mounted without a log device; be sure to create the directory it is to be mounted on first:

# mkdir /usb_mnt

# mount -o log=NULL /dev/usbms0 /usb_mnt

# df -g |grep usb
/dev/usbms0        3.75      3.73    1%        4     1% /usb_mnt

If you wish to use a log device to mount the usb stick, use the inline log. This ensures that it is contained within the file system:

# mkfs -olog=INLINE,ea=v2 -Vjfs2 /dev/usbms0
mkfs: destroy /dev/usbms0 (yes)?
logform: Format inline log for  <y>?y
File system created successfully.
3912376 kilobytes total disk space.
Device /dev/usbms0:
  Standard empty filesystem
  Size:           7824752 512-byte (DEVBLKSIZE) blocks
# mkdir /usb_mnt
# mount -V jfs2 -o log=/dev/usbms0 /dev/usbms0 /usb_mnt

Unmounting a memory stick

Once you have finished using the a USB stick file system, unmount it, remove it, then delete the usbms0 device:

# umount /usb_mnt
# rmdev -dl usbms0

Data on the stick

In this demonstration, I am using a CSV text file called plaks.txt. It is an extract from a database, partial contents of the file is shown below:

alpha,uk01w,12001,jan,-2
bravo,ge01w,98801,jan,-3
charlie,se01w,98111,jan,0
delta,my01w,18811,jan,4
echo,sg01w,34131,janq,2

Earlier I suggested that reading from a USB stick is quick but writing to the USB stick is slower compared to internal disks. This can be somewhat verified by doing an example copy. First, I copied a file from the USB stick to an internal disk. Then, I copied the same file from the internal disk back to the USB stick using the timex command to show the timings of the copy commands. Before both copy operations are carried out, the file systems have been unmounted and re-mounted, so that the operation does not use the file system cache and thus does not blur the timings.

First, the file is copied from the USB stick onto an internal disk to the /holding directory, which resides on an internal disk:

# pwd
/usb_mnt
# timex cp plaks.txt /holding/
real 0.03

Next, I remove the file from the USB stick and unmount and remount the file systems.

Next, I copied the file back from /holding to the USB stick.

# cd /holding
# timex cp plaks.txt /usb_mnt/
real 0.06

The copy of the USB stick takes nearly half the time of the copy from an internal disk. I typically use the USB stick for text extraction. Using awk as an example on data extraction, I extract the pattern 'bravo' contained in the plaks.txt located on the USB stick:

# pwd
/usb_mnt
# timex awk '/bravo/' plaks.txt
real 5.50

Doing the same operation on an internal disk:

# pwd
/holding
# timex awk '/bravo/' plaks.txt
real 8.48

The text extraction is quicker on the stick; in fact, there is a difference of over 3 seconds. Here I am using a relatively small size text file. Though clearly the saving in time is greater when using larger sized exported file, which typically could be up to 8-10MB.

The above, in my opinion, justifies why I choose to use USB sticks for heavy test processing; however, as suggested earlier, if you need to write data you lose out big time. Here I create a 100MB file on the USB stick:

# pwd
/usb_mnt
# timex lmktemp myfile 100M
myfile
real 6.60

Next, I do it on an internal disk:

# pwd
/holding
# timex lmktemp myfile 100M
myfile
real 1.65

As you can see, there is a difference of over 5 seconds in the writing to a USB stick compared to an internal disk.

USB sticks are great for transferring data when there is no real alternative. I recall one incident, where one of our machines located in a remote region where the SAN disks were failing. These SAN disks contained work flow images. This machine did not have a sturdy network, and we wanted to get these images onto another remote machine for business users to be able to continue to work. The only method quickly available was to buy some 64GB USB sticks, mount them, and tar the images up on the USB sticks. The images were then extracted onto our other AIX box. This method worked with no data loss.


Mounting a previous initialized memory stick

If you are in possession of a memory stick that contains data what was previously mounted as a file system, to access it simply mount it. Be sure to create the mount point first. For example, assume the USB device is inserted in the second USB slot, then have the device discovered as usbms1:

# mkdir /usbstick
# mount -o log=NULL /dev/usbms0 /usbstick

Conclusion

Using USB sticks is a great when you want to create a fast read access scratch file system for processing or maybe a raw device to dump some files into for transport to another AIX system. USB sticks offer one solution where you have a poor network, and you need to get a large amount of data transferred.

Resources

Learn

  • The AIX 7.1 and AIX 5.3 Information Center is your source for technical information about the AIX operating system.

Get products and technologies

  • Try out IBM software for free. Download a trial version, log into an online trial, work with a product in a sandbox environment, or access it through the cloud. Choose from over 100 IBM product trials.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into AIX and Unix on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX
ArticleID=768164
ArticleTitle=AIX and USB memory sticks
publish-date=11012011