Add Linux users with a crypted password

If you routinely setup Linux systems with the same user accounts, you might find it helpful to be able to add the users with a single command, without typing in the password and other info for each user like you have to do with adduser. A single-line, no-questions-ask command also lends itself well to batch scripting.

To do this you'll need to know the username, the system user id and group id (UID and GUID) and plain text password for the user you're about to add.

Start by making the crypted password hash. You can make the password hash with the following command (this is tested on Debian Linux):

    mkpasswd -H md5

mkpasswd will ask you for the password:

    Password:

Type in your password, hit enter and mkpasswd will show you the hash:

    $1$uv.y5wtb$remRyh2SeDD9mgZ81aYuB1

Here is the full command. Put the hash mkpasswd printed in single quotes at the end of the line like this:

    useradd -g 1003 -m -u 1003 -s /bin/bash johndoe -p '$1$uv.y5wtb$remByh2ShDD9mgZ81aYuB1'

The shell parameter "-s /bin/bash" is not strictly necessary but I've found it best to include it to avoid potential accidents.

Note that when using useradd like this, the groups must be preexisting. If the group doesn't already exist, add it with:

    groupadd groupname

It's a good idea to clear your shell history after adding all your users this way. An easy way to clear the history is by running the following commands, one at a time:

    HISTFILESIZE=1
    ls
    ^d

(^d means Ctrl-d: hold down the control key, tap the d key, then let go of the controll key)

Emacs Example Tutorial: replace words text with a regexp

In Emacs' normal find-and-replace mode you can't replace special characters like "beginning of line" or "end of line" with anything because it treats the special characters representing end of line (^ and $, respectively) as regular characters - a literal ^ and a literal $.

If you want to replace the beginning of line or end of line with another string (and this can come in extremely handy some times) you have to use a different kind of replace: RegExp replace (regexp stands for the term regular expression, which is a phrase used to describe computer pattern matching code or directions. Think "filters"). To get Emacs into this mode type:

M-x replace-regexp

Remember, M is the meta key, which is usually equal to holding down the alt key or tapping on the escape key. So you can think of this as:

ALT x

Emacs will prompt you that it is waiting for more instructions by displaying "M-x". Then you type in:

replace-regexp

and hit enter. Emacs will now ask you "Replace regexp: " - it is waiting to know the regular expression that matches the characters you want to replace.

So let's say you have a file of information - a list of names of employees, for example. This hypothetical list contains nothing but their names. Let's say you want to add a note at the end of the line that reads "Note: give this employee a new mouse pad:."

While emacs is sitting there asking you "Replace regexp: " type in just the dollar sign and then hit enter: $

Now Emacs will ask you "Replace regexp $ with: "

You can now type in any text you want, and then hit enter - Emacs will append that text to the end of the line. Unlike the regular find-and-replace, replace-regexp does not ask you for confirmation one line at a time, it will simply apply this "replacement" to every line of the file at the end-of-line mark.

As a very cool side benefit, you can combine this with block editing mode to constrain the replacement to a fixed block of lines. For more on block editing mode and how to define a block, see this article on emacs block editing.

Emacs Example Tutorial: replace words in a file

This example shows how to search-and-replace words in a file.

First we assume you are used to searching for strings or characters in a word, if you are not, please see this article on how to use emacs to find words and letters in a file.

You can replace strings or characters (words, letters) in a file in a manner very similar to find. You type this command (remember, M is the meta key, which is usually equal to holding down the alt key or tapping on the escape key):


    M %

So you can also think of this as:

    Alt Shift 5

After you type that key command sequence, Emacs will ask you:

    Query replace:

Then you type in the characters that you are looking for that you want to replace and hit enter. Then Emacs will ask you:

    Query replace [search-string-here] with:

Now you type in the characters you want to insert in place of the old word and again hit enter. Emacs will find the first instance of the "search" string (the word you want to replace), it will highlight that occurrence and then ask you:

    Query replacing [search-string-here] with [replacement-string-here]: (? for help)

If you want that occurrence to be replaced, you just tap the y key for "yes". Emacs will automatically move on to the next occurrence of the string you searched for and ask again:

    Query replacing [search-string-here] with [replacement-string-here]: (? for help)

If you want to skip any particular occurrence, just tap the n key for "no". Emacs will continue find and replace until you reach the bottom of the file.

Emacs Example Tutorial: find words and letters in a file

When using Emacs probably my #1 most used and favorite feature is to "find" or search for a string - strings are groups of characters that may or may not be part of a word, a whole word, or a sentence (to be exact there is no maximum length that I know of).

Once emacs is open just hit Ctrl-s (Hold down the Control Key and while holding it, tap the s key once, then let go of Ctrl). Many people will recognize this as the typical "save" command many graphical application use to save a file. Don't worry, emacs has a different save command [link]. After you have hit Ctrl s, emacs will wait for you to start typing the characters (letters) you are looking for. It sill search "as you go", meaning that if you type "f" it will jump right to the first "f", then if you type "i" it will move to the first occurrence of "fi", then if you continue typing "nd" it will move to the first occurrence of the word "find". If it does not find the character sequence you've typed in, it will tell you so.

After it finds the first occurrence of your string of characters, you can hit "Ctrl s" again to find the next occurrence of that same string in your document. You can continue hitting "Ctrl s" until you get to the end of your file, at which point Emacs will tell you "Failing I-search: [search-string-here]" which means it has hit the end of the file. If you continue taping "Ctrl s" even after that, Emacs will wrap your search back around to the start of the file and continue searching from the top again. You will know it has gone back to the top of the file when you see Emacs display "Overwrapped I-search: [search-string-here]" at the bottom of the window.

Emacs Example Tutorial: exit emacs and save a file or discard changes

For those who are just starting to learn emacs, the key command sequence to save a file is:

Ctrl x s

That means hold down the Control key, tap the x key once, and the s key once, let go of Control key. If you're used to using "Cntl+s" in a graphical progam to save file, this key binding might seem a little awkward at first but you'll get used to it quickly.

You can also choose whether or not to save by simply exiting emacs and telling it "y" for "yes" or "n" for "no" when it asks you if you want to save your changes (also known as "Exit saving changes") by typing the "quit" key command sequence:

Ctrl x c

Emacs will then prompt you "Save file /path/to/filename? (y, n, !, ., q, C-r or C-h)" which means you must answer with one of the options listed inthe parenthesis. To save just tap the "y" key and you're done.

If you decide you don't want to save your changes, tap "n" for "no". Emacs will then double-check your decision by asking you:

Modified buffers exist; exit anyway? (yes or no)

Type "yes", which in this context means "yes I really do want to exit and discard my changes"
If you type "no" emacs will send you back to editing mode with Emacs still open and your unsaved changes still visible and accessable in the window.

Installing Mac OS X software updates from the commandline

I don't know why I didn't know this existed earlier - it's the most helpful computer administration command I've found in a year.

Apple's Software updates can be downloaded and installed on Mac OS X with the command "softwareupdate". This makes Updating your OS X remotely via ssh a snap compared to using ARD or VNC.

The command:

softwareupdate -l

will scan the apple update site for pending updates, and display them complete with a commandline-friendly label (name).

You can then install the updates you want with

softwareupdate -i [update_label]

For example:

$ softwareupdate -l
Software Update Tool
Copyright 2002-2005 Apple

Software Update found the following new or updated software:
   * JavaForMacOSX104Release7-1.0
      Java for Mac OS X 10.4, Release 7 (1.0), 82580 [recommended]

$ softwareupdate -i JavaForMacOSX104Release7-1.0

The command output shows which upates are recommended and which will requite a system restart.

The install command prints out a handy character based progress meter while it's downloading and installing:

Downloading Java for Mac OS X 10.4, Release 7
 Downloading Java for Mac OS X 10.4, Release 7
 0..20..40..60..80..100
 Expanding Java for Mac OS X 10.4, Release 7
 Installing Java for Mac OS X 10.4, Release 7
 0..20..40..60..80..100
 Done.

This is one terrific feature!

Emacs Example Tutorial: insert a block of text

As I explained in my previous Emacs tutorial on killing a block of text, it is sometimes hard to find easy to understand documentation on how to use emacs.
One of the first things I wanted to learn how to do in Emacs was block editing. By "block editing" I mean select a vertical area of text (one or more columns across multiple rows making a square or rectangle - you might call it zero or more since you don't actually have to select any characters, only the spaces between them) and do things to it - you might call it column editing or column mode or "Block Mode Editing" like in Kate, or UltraEdit.

Inserting a column of characters in emacs (without using a mouse) is really easy: you start by highlighting or selecting an area - when you do this you are said to "mark" the region.

You mark a region by positioning your cursor in one corner of the area you want to mark and hitting Control-Spacebar. You will see Emacs respond with the message "Mark set". Then navigate your cursor (I always use the arrow keys, but there is probably other ways to do it) to the opposite corner of the region you want to mark. The "region" is now marked as a rectangle between wherever you started (when you hit Ctrl-Space) and wherever your cursor is now sitting. Note that you don't do anything special to mark the end of the region -- the end is where ever your cursor is positioned.

Emacs calls the space you've now selected the "region-rectangle". Once it's highlighted, you can insert characters into the region by typing:

C-x r t

(That means press x while holding down the control key, release both keys, press and release the r key, press and release the t key). Emacs will respond with the message "String rectangle": it is asking you what text you wish to insert. Type what ever characters (letters) you want to "paste" into the region, and hit enter. It will automatically fill in that same text on every line that you've selected in your region.

I find this to be a great way to comment out areas of code with # characters. Emacs also has a "comment region" command which sometimes works, but occasionally emacs is too smart for it's own good and doesn't know what comment character to use (in which case it will usually ask you which to use with the question "No comment syntax is defined. Use:"), or worse it will use the wrong one.

You can use the comment region function by selecting a region in the same manner descried above, then type:

M-x comment-region

(That means press x while holing down the ALT key, release both keys, type "comment-region" - without the quotes of course - and hit enter).

I find C-x r t easier and faster to use than "M-x comment-region" in most cases anyway. It's fewer letters to type, even when you consider tab completion.

Linux on a Dell Vostro 1710 laptop - don't try it [yet]

Recently my company needed a new laptop for one of our employees. After my great success with installing SuSE Linux on a Dell Vostro 1700 laptop, we decided to buy a Dell Vostro 1710, hoping that it would have better, newer hardware and still work with SuSE.

Wrong.

For this first time in my entire 11+ years of using Linux, the installer could not use the trackpad or even the keyboard. Yes, you read that right. I was able to complete the install with an external USB mouse and keyboard, but after the install finished and I rebooted the system without the peripherals (hoping that it was only the installer having trouble), the laptop's built-in keyboard and mouse still wouldn't work. I can't image what Dell has done to prevent detection/use of a simple keyboard.

I never bothered to try the other hardware (camera, etc) - we sent the 1710 back and bought a refurbished 1700.

I strongly suggest nobody buy a 1710 until at least another round of distros has been released this coming winter or spring. It was openSuSE 11.0 that I tried on the 1710 - perhaps 11.1 will work with the 1710, but I'm sure I won't be ordering another one to try it on.

Setting the computer hardware clock (BIOS or CMOS) from inside a running Linux system

Normally after setting the date and time in Linux (for example using the NTP service or ntpdate, you can set the hardware clock to match with the following command:

hwclock --systohc

That way when you reboot, your clock will still be correct (or close to it).

I've discovered that this method does not work on the newer Dells, such as our 1U PowerEdge 1850s. Supposedly this has something to do with the kernel RTC module, and hardware changes. However, the following command appears to set the clock just fine on the problematic Dell servers:

hwclock --systohc --directisa

"I keep just enough vi knowledge in my head so that I can edit a Makefile and build Emacs." -- Tony Foiani, 1999

How to enable the root user in Mac OS X using NetInfo, or change root's password:

1. Open the NetInfo Manager utility (located under /Applications/Utilites/).
2. Click the lock (padlock icon) in the NetInfo Manager window.
3. Enter an administrator account name and password, then click OK (this isn't Administrator as in root, it's Administrator as in a user account with administrative privileges, typically the first account created during the initial install or setup).
4. For Mac OS X 10.2 and later, choose Enable Root User from the Security menu.
5. If you have not previously set a root password, an alert box may appear that says "NetInfo Error," indicating that the password is blank. Click OK.
6. Enter the root password you wish to use and click Set.
7. Enter the password again for verification and click Verify.
8. The root user is now enabled.
9. Click the lock again to prevent changes.
   

Social networking

Today I'm trying out a Technorati Profile to see if any of these things actually help. Still got lots to learn! :-)

Linux on a Dell Vostro 1700 laptop

At my company, Cedar Creek Software, we recently purchased a new laptop:

Dell Vostro 1700 laptop
Model # PP22X
17" wide aspect-ratio screen

We use SuSE Linux on our workstations, because it "just works" to a greater degree than any other distro I've tried in the last 9 years - at least of all the distros that are still ticking (and I've tried quite a few).

lspci -nn gives the following specs for the monitor:

01:00.0 VGA compatible controller [Class 0300]: nVidia Corporation GeForce 8600M GT [10de:0407] (rev a1)

And the following for the 4965AGN wireless network adapter:

0c:00.0 Network controller [Class 0280]: Intel Corporation PRO/Wireless 4965 AG or AGN Network Connection [8086:4229] (rev 61)

At first I tried SuSE 10.2 and was able to get the monitor working with the usual pre-built nVidia kernel modules from nVidia's repository, but I could not get the wireless adapter to work. Everything else worked fine.

For the wireless, I tried installing the official Intel drivers but I couldn't make it happy with the SUSE-supplied kernel and kernel sources. It kept complaining about the mac80211 subsystem, and many other things. I tried a lot of stuff, including a vanilla kernel, which the Intel drives were unable to patch and use properly (I have not idea what I did wrong). I tried NDISWrapper. Contrary to what it says on the en.opensuse.org wiki, I did get NDISwrapper to work with the Windows XP driver, but it made the laptop unstable - it would just freeze up randomly and I'd have to cycle the power.

Finally I upgraded to a kernel & sources from the SuSE kernel-of-the-day repository (which now seem to have a good set of mac80211 modules sources and all the other things that are needed to support the drive from the Intel wireless project), but then when I'd try to run iwlist and iwconfig it would complain that the module (or something) was complied with support for a higher version of something (I forget exactly what) than the wireless tools.

One of the things I love about SuSE is their hardware support. New hardware is often supported by the very next release of SuSE, however the next version of SuSE - 10.3 - isn't out yet. But it's getting to the final stages of beta testing (thank you, Novell / openSuSE, for the new open development model that gives us access to betas!), so I downloaded the latest LiveCD made from the 10.3 beta tree and discovered that the 4965AGN wireless card worked in the LiveCD environment just fine.

So I've downloaded and installed the RC1 release of SuSE 10.3 on the Vostro 1700. They've made some small improvements to the installer and and distro and I really like it. SuSE is such a fine distro, I strongly recommend it for workstations, laptops, and personal computers.

The troublesome 4965AGN was immediately detected by the installer - Yay! - and works great with no trouble.

The nVidia GeForce 8600M GT graphics card was correctly detected and configured at the optimum size by the 10.3 installer/config tools, unlike 10.2 which left the config set to a low resolution by default. That's with the xorg open source drivers that came with SuSE, of course.

However I did have some trouble configuring the dualhead setup. I needed to be able to use a secondary monitor on the laptop's VGA out as a second screen. I usually use Xinerama, configured with SaX2, for this kind of setup.

I downloaded the latest source driver from nVidia (it's not really all source, but it's the package nVidia provides that allows you to build your own kernel module) and it compiled quickly with no trouble. The documentation on the opensuse.org wiki then says to then use "sax2 -r -m -0=nvidia" to configure the monitor. That works fine for the laptop's own monitor, at 1440x900, but when I tried to use SaX2 to configure Xinerama for the second monitor, it wouldn't work at all. The resulting X config left the laptop's screen turned off and put the laptop screen's setting on the external monitor - which of course were the wrong size, shape and resolution. I spent a good amount of time trying to make this work, then read about using "TwinView" without Xinerma in the nVidia documentation. I tried it out and it works. I tried using the nVidia-supplied config tool, nvidia-xconfig for both the laptop-only config and the dual monitor config and it produced perfect, working configs for both. I highly recommend it.


For the dual-head config just plug in the external monitor and run (as root):

    nvidia-xconfig --twinview

Emacs Example Tutorial: kill a block of text

When trying to learn to use Emacs, I had a very hard time finding out how to kill a multi-line block of text in Emacs. When I say "block" I mean a square or rectangle that does not include anything outside the marked area, in other words, not including the rest of the lines - whatever portion I didn't mark. For example, the equivalent of "Block Mode Editing" like in Kate, or UltraEdit.

If you go search on google for this kind of block "cut", you'll normally find documentation telling you to use C-w (note to newbies: in emacs documentation when you see "C-", that means the Control key, and when you see "M-", that means the Meta key, which is the "Alt" key for most PC users. So C-w is Control-w). This kills all text on every line, including the end of the line, within the whole region, from the Mark to the Point. When I am trying to trim out a block of text from multiple lines (such as an unusually large comment block, or a portion of a column from a tabulated text file), I don't want the whole line killed, just the letters and other characters inside the square block.

I finally found that this is called killing the "region-rectangle", and the command is:

    C-x r k

For those that don't know: you "mark" a region in Emacs by positioning your cursor in one corner of the area you want to mark and hitting Control-Spacebar. You will see Emacs respond with the message "Mark set". Then navigate the cursor (I always use the arrow keys, but I guess there's other ways to do it) to the opposite corner of the region you want to mark. The "region" is now marked as a rectangle between wherever you started (when you hit Ctrl-Space) and wherever your cursor is now sitting.

Deleting files based on content: using find, grep, xargs and rm

Today a user asked me to help remove a deluge of spam from their inbox, since they had been flooded by a particular spammer. Well, everyone makes mistakes sometimes and today was my day: I made a mistake with -print in the find command and accidentally deleted all of the users's email. Whoops.

All the spam contained the the word "Desv" as part of the from. I could have used grep, but there were so many files in the directory that grep -i Desv * complained that the argument list was too long - this is a nice "safety" feature of bash that wasn't always there. Someone figured they should limit the shell globbing of * so it couldn't generate a gigantic list that could cause buffer overruns. So I had to use find to get around the file glob size restriction. The right syntax for find turned out to be:

    find . -exec grep -l Desv {} \; | xargs rm

Simple Newbie's Guide to netstat

I've been using Linux for 8 or 9 years now and I have always heard that, in order to find out what program or process is using a given open port on your computer you should use the program "netstat".

Whenever I've asked just exactly how one uses netstat, I've been told "netstat -a".

Now, for those that think that's the right answer, please go try "netstat -a" and see just exactly what it tells you. Can you use that information to find out what program or process is holding that mysterious port open? Uh-huh. I thought the same thing the first time I tried it.

I have essentially ignored netstat and worked around it, because it's man page is one of the better examples of a wonderfully esoteric but not-too-helpful man page: it lists paragraph after paragraph of command options, leaving the average newbie's mind in a rather confused state.

Today I took the time to read the long-winded man page and found that netstat really can tell me what I want to know:

-v Gives "verbose info" (of course), though I didn't find this very handy.
-l Shows only listening sockets. This is not a default option, and you really want to see the information -l has to show you.
-e Display additional information.
-p Show the PID and name of the program to which each socket belongs.

-p is the real winner here: it shows you the process number of the program that's using this port, and the name of the program. This will enable you to find the process in ps, and/or kill that process.

Since "netstat -a" puts out a whole lot of info, I found that starting with "netstat -lt" and "netstat -lu" is a great first or second step for finding your open ports (the other one being port scanning, which you really should do anyway). Once you find a port you want to close, you can run "netstat -aevp" and grep for the port number (or port name as found in /etc/services) you are interested in to find the program's PID and name (netstat -aevp | grep 59849).

Give it a try:

netstat -lt
netstat -lte
netstat -ltv
netstat -ltp
netstat -aevp
netstat -aevp | grep sunrpc
netstat -aevp | grep domain

Installing non-free Sun Java on Debian

There seems to be some confusion (at least for some folks) about what it takes to install Sun's Java on Debian. It is against Debian's policy to distribute "non-free" packages in the main tree, however, that's exactly what the "non-free" repository is for.

Using the Debian web site's package search will quickly show that they do, in fact, offer "commercial" java packages.

For etch 4.0 (stable), just add "non-free" to your /etc/apt/sources.list:

deb http://ftp.us.debian.org/debian/ etch main contrib non-free

Run (as root, of course) "apt-get update" then "apt-cache search java2" and you'll find:

sun-java5-jdk
sun-java5-jre

Currently they are version 1.5.0-10-3.

To install them run:
    apt-get install sun-java5-jdk sun-java5-jre

Passwordless log in with SSH keys

    A very fine feature of UNIX operating systems is native support for services like SSHd. SSH provides shell access on remote computers - encrypted, so no one can snoop your passwords. SSH comes with various utilities including scp, which allows you to copy files and folders to the remote server (and all this traffic and data is encrypted). You can also use ssh as rsync's transport mechanism for synchronizing directories across computers. If you want to automate tasks like this (scheduling unattended backups with cron, for example), you'll need the computer that's initiating the connection to be able to log into the remote computer without being prompted for a user to input a password.

This is where ssh 'keys" come in. You create a private/public key pair on your computer ("the client"), and put the "public" key up on the remote computer ("the server"). After that, when you log into the remote computer, you are no longer prompted to enter a password (so long as you made the key correctly).

To generate a pair of keys with no passwords, using the standard RSA type of keys, run the following command on the computer you are connecting from:

    ssh-keygen -t rsa -N ""

This will create two files: ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub
Copy - with scp - the public file ~/.ssh/id_rsa.pub up to the remote computer you will be connecting to (or sending data to). Then log into the remote server and put the contents of that public key into the ~/.ssh/authorized_keys file. An easy way to do that is like this:

    cat >> id_rsa.pub ~/.ssh/authorized_keys

Using the double "greater than" brackets ensures that if ~/.ssh/authorized_keys already exists, the new key will be added to the file, and not overwrite any existing keys that might already be in place.

After that, you should be able to connect from the client with the key pair to the remote server without being prompted for a password.

Mac OS X Keyboard Shortcuts

    Apple provides a nice list of keyboard shortcuts for OS X:
http://docs.info.apple.com/article.html?artnum=75459

My favorite: "Shift-Command-Delete": Empty Trash

Reporting ebay scam email

     I don't know if it does any real good or not, but I've taken to forwarding all ebay spoof/scam/phishing email to spoof@ebay.com (which gives me a nice little automated reply stating "This message did not originate from eBay . . .") At least that way eBay will have more recorded instances of such emails in case they ever file legal proceedings against these scammers.