March 31 is [World Backup Day]!
Recently, a client asked how to backup their IBM PureData System for Analytics devices. IBM had [acquired Netezza in November 2010], and later renamed their TwinFin devices as the IBM PureData for Analytics, powered by Netezza.
The [IBM PureData System for Analytics] is incredibly fast for performing deep, ad-hoc analytics. However, the people who use them are "data scientists", not backup experts.
Likewise, there are backup administrators who may not be familiar with the unique characteristics of this expert-integrated system to know what backup options are available.
As with the rest of the IBM PureSystems line, the IBM PureData System for Analytics (or, PDA for short) has a combination of servers, storage and switches inside.
In a full-frame PDA, there are two servers in Active/Passive mode, these coordinate activity to FPGA-based blade servers, which have parallel access to hundreds of disk drives, storing nearly 200 TB of compressed database data. A system can span up to four frames.
But what do you backup? And why? You don't need to worry about backing up the Linux operating system or NPS server code, that is considered firmware and if anything every got corrupted, IBM would help restore it for you. System-wide metadata, such as the host catalog and global users, groups, and permissions should be backed up periodically to protect against data corruption.
There are a number of reasons to backup your user databases:
The PDA has three backup formats. You can backup the entire user database in compressed format, backup individual tables in compressed format, or export to a text-format file.
Compressed format is faster, but can only be restored to the same PDA, or a PDA that has the same or higher level of NPS firmware. The text-format is slower, but can be used to restore to lower levels of NPS firmware, or to other database systems.
There are basically two methods to backup your PDA. The first is called the "Filesystem" method. Basically, you can attach an external storage device to the NPS server, and use the built-in command line interface (CLI) to store the backups onto its file system.
You may find that your databases are so large, they will exceed the limits of the filesystem on the external storage device. For SAN or NAS deployments, I recommend the IBM Storwize V7000 Unified with IBM General Parallel File System (GPFS). However, if you are using something else, you may need to use the "nz_backup" scripts provided which split up the backup images into smaller pieces that most other filesystems can handle.
The PDA comes with 10GbE Ethernet ports that you can attach a NAS storage device over a Local Area Network (LAN), or add Fibre Channel Protocol (FCP) ports and connect over a Storage Area Network (SAN). To keep things simple, I will refer to whichever network you decide as the "Backup Network" in the drawings.
The second method for backup is called the "External Backup Software" method. As you have probably guessed, it involves sending the backups to a supported software product like IBM Tivoli Storage Manager (or, TSM for short).
In this case, the PDA acts as a client node, similar to a laptop, desktop, or application server with internal disk. Backup data is sent over the LAN to the designated TSM server, and the TSM server in turn writes over the SAN to its storage hierarchy of disk, virtual tape and/or physical tape resources.
Backups can be done by command "on demand", or automated on a schedule. For the /nz/data directory, direct the nzhostbackup command to send the backup copy to local disk, then use TSM's dsmc archive command to transfer this backup copy to the TSM server.
For nzbackup with the users or db parameters, you can send the data directly to the appropriate TSM server by specifying the connector and connectorArgs parameters.
To reduce traffic on the TSM Server, an intermediary "TSM Proxy Node" can be put in between. In this case, the PDA sends the backup to the Proxy Node, the Proxy Node uses a "LAN Free Storage Agent" to send the backups directly to the virtual tape and/or physical tape, and then notifies the TSM Server to updates its system catalog to record which tape holds these new backups.
Another configuration involves installing the TSM LAN Free storage agent directly on the PDA. While this will require FCP ports to be added and consume more CPU resources on the NPS server, it eliminates most of the LAN traffic, allowing the PDA to send its backups directly to virtual or physical tape.
To learn more about this, see my full presentation [Backup Options: IBM PureData System for Analytics, powered by Netezza] on the IBM Expert Network powered by SlideShare, or attend the upcoming [IBM Edge 2014] conference in Las Vegas, May 19-23. I will be there!
technorati tags: IBM, Netezza, PureData, PureData for Analytics, PDA, World Backup Day, Backup, NPS, nzhostbackup, nzbackup, expert-integrated, Tivoli, Tivoli Storage Manager, TSM, dsmc, #ibmedge, Slideshare
My how time flies! It has been nearly a year since our new Tucson Executive Briefing Center had its [Ribbon Cutting Ceremony].
To celebrate this achievement, IBM asked me to write and direct a short film to remind everyone we are here to help clients solve problems, determine an appropriate strategy and make solid purchase decisions.
I have produced other videos for IBM. See my October 2013 blog post [Incorporating Videos] for other examples. This was my first time as writer/director for a project.
This video won't win any Oscars, but I would still like to thank the Academy, my colleagues IBM VP Calline Sanchez, Lee Olguin, Joe Hayward and Kris Keller agreeing to be filmed on camera. Behind the scenes, I want to thank IBM Fellow John Cohn for his superb narration, Andrew Greenfield as cinematographer and editor, Shelly Jost as creative consultant selecting the musical tracks, and Denise White for reviewing the screenplay. Finally, I want to thank our producer, Bill Terry, for funding this effort.
What do you think? Will it go viral? Enter your comments below!
IBM Cloud announcements at Pulse 2014
Well it's Tuesday again, and you know what that means? IBM announcements! Many of the announcements were made by IBM Executives at the [IBM Pulse 2014 conference].
I am not at Pulse 2014 this year, but I managed to watch many of these announcements on the [IBM Pulse livestream].
Continuing my series on building a Desktop computer for a kindergarten class, I look at Fedora with Sugar mentioned in the article [Top 6 Linux Distributions for Children (Ages 2 and Up)].
(This series started with my post [Kindergarten desktop - The Challenge]. I have a 512MB RAM system with 40GB disk drive that I will install Linux and educational software for a class full of kindergarten children. My previous post covered three other Linux distributions [LinuxKidX, Qimo, and Foresight for Kids].)
I am not stranger to the Sugar learning platform, developed as part of the One Laptop per Child [OLPC] project.
As I mentioned in my post [Helping Young Students - part 1], I was part of the OLPC development team back in 2008, helped local volunteers deploy laptops to children in Nepal and Uruguay, mentored a college student in India, and learned a lot of Python programming language in the process.
Sugar is now developed by Sugar Labs, a nonprofit spin-off of OLPC. The code is free and open source desktop environment for many other machines, including as a "Desktop Environment" for Fedora Linux.
I kept my 40GB hard drive partitioned as follows. On the extended partition, sda5 will hold my system utilities, like Clonezilla and SystemRescue, and sda6 is my swap space, increased to 1500MB. Partition sda1 has Edubuntu 12.04 on it, and I will use sda2 to install Fedora with Sugar.
[Sugar-on-a-stick], is so named because it is designed so that each child has their own LiveUSB. This can run on PC with Windows or Mac OS without affecting those operating systems, allowing a child to use Sugar in the classroom, then take the stick home and continue on their home PC.
A 2GB or greater USB memory stick can hold both Fedora and Sugar, and use that to boot your desktop. Unfortunately, it requires 1GB of RAM, and I have only 512MB. Can I just run Sugar natively on a Fedora install? Yes, thanks to the [Sugar not "on a stick"] instructions, I can install Fedora first, then just:
$sudo bash #yum groupinstall "Sugar Desktop Environment"
Unfortunately, the latest Fedora release (F20) recommends 1GB of RAM. Fortunately, I found Dean Howell's rant [Fedora Irresponsibly Lowers Memory Requirement To 512MB] about the Fedora F17 release. I gave this a try.
There are three ways to install Fedora:
I chose method 3 and downloaded the appropriate ISO file. While F17 only requires 512MB of RAM to run, the graphic installer requires 768MB, and is fully explained in this [29-step F17 installation guide].
To get around this, select "Troubleshooting" which then lets you select low-graphics/text mode installation that ran well under 512MB. I installed both LXDE and Sugar, and everything worked fine!
Why both LXDE and Sugar? Well, Sugar is quite a different environment, and I wanted LXDE as an alternative for the admin and teacher to use.
The article on [Sugar software on Wikipedia] sums it up well:
"Unlike most other desktop environments, Sugar does not use the 'desktop', 'folder' and 'window' metaphors. Instead, Sugar's default full-screen activities require users to focus on only one program at a time. Sugar implements a novel file-handling metaphor (the Journal), which automatically saves the user's running program session and allows him or her to later use an interface to pull up their past works by date, activity used or file type."
Now that I have that working, it is time to upgrade from non-supported F17 to a supported level. Ravi Saive explains the [Four Ways to Upgrade from Fedora 17 to Fedora 18]:
As you can probably guess from the title of this post, I chose method 2 "FedUp" as it seemed to be the least invasive. I was unsure if method-1 "Clean Install" of F18 would work with 512MB of RAM, and I have been through enough horrors of failed yum upgrades on my own Red Hat Enterprise Linux [RHEL] at work to avoid method 3. Method 4 is just a script to automate the steps of method 3.
The steps are fairly straightforward. First, install the FedUp package, run "yum update" to ensure you have all the latest kernel and F17 packages for everything else, and reboot.
Then run the fedup-cli command, which upgrades all the packages to F18 level and creates a special kernel level that will then finish the install after the second reboot. It took a while, so I let it run unattended. I put the debug log on partition sda5 in case anything went wrong.
#fedup-cli --reboot --network 18 --de
What could go wrong? Well, it turns out that fedup works by updating the Grub2 boot loader configuration, but my grub2 resides on sda1 partition instead, owned by my existing Edubuntu. The reboot did not give me the option to run the specialized kernel to finish the process.
Fixing this was a hot mess, but I managed to configure Grub2 on Fedora, and complete the upgrade and get everything working as before. However, even though it just came out last year, [F18 version is already out of support]! This means I get a second chance to do FedUp, this time to F19 release. Oh boy! Fun!
While the second time went smoother, the problem was that F19 doesn't seem to run well in 512MB of RAM, and chances are F20 won't either.
So what have I learned from this?
If you have any experience with Fedora or Sugar in the classroom, comment below!