Mastering The Command Line

Archive for Mastering The Command Line

man Pages for C Development in Ubuntu

by Chris Jean

September 26th, 2009

I recently tutored a friend in C coding. Since I hadn’t worked with C in at least 8 years, I really needed to have some references to rely on for syntax and other specifics. Fortunately, there are some easy man pages that can be installed in Ubuntu that offer helpful information that I was able to use to help refresh my memory.

These man pages are easily installed by installing the manpages-dev package. You can install this package via Synaptic or directly on the command line. I like the command line method personally, so I ran sudo apt-get install manpages-dev from the command line to quickly install the package.

After installing the package, I’m able to access man pages for functions such as printf, opendir, and putc. For each function, it shows the valid syntax as well as what library is required to make use of the function.

The information isn’t limited to functions as you can also access information on the libraries, such as stdio or string.

To access any of this information, simply run man [function or library name] such as man stdio.

This package isn’t limited to C functions/libraries. It is a general use Linux development suite of man pages. For a full list of what is installed, check out the file list.

Categories Development, Linux, Tips 'n Tricks
Comments (0)

Updating Multiple Git Repositories Easily Using Bash for Loop

by Chris Jean

September 15th, 2009

All of the WordPress themes that I work on for iThemes are managed as Git repositories. Recently, we moved past the 100 repositories mark. That’s a lot of repositories to manage, and unfortunately, too many of those repositories contain duplicated information.

Later on, I might delve into how we use Git to manage our theme repos. For today, however, I’d like to focus on how I quickly and easily pushed up changes to more than a dozen repos in a single, albeit long, Bash command.

I had finished making updates to 16 Flexx repos, and I needed to push all of those changes up. Since I had multiple working repos in that folder, I was lucky that each of these repos began with the text “Flexx”. Also, since they are all part of the same series and need to keep the same version number, that simplified the tagging as all could be tagged as 2.5.0.

Given this information, I simply ran the following command from the directory that contained all the repository directories:

for i in `ls|grep Flexx`; do echo "--- Pushing $i"; cd $i; git commit -am '2.5.0' && git push && git tag 2.5.0 && git push --tags; cd ..; echo "--- Finished $i"; done

There’s a lot going on here, so I’ll break it up and explain what I’m doing.

Categories Development, Linux, Tips 'n Tricks
Comments (0)

Command Line History in Ubuntu Terminal

by Chris Jean

March 9th, 2009

I’ve had a lot of fun recently posting about how to do stuff on the command line in Linux. My focus is specifically for Ubuntu users, but the information and techniques can be used for any Linux distro.

Since I’m probably going to end up with a lot of content under this topic, I’ve decided to create a dedicated tag: Mastering the Command Line. I’ve gone through my older posts on this topic and tagged them as well. So, make sure to check out Mastering the Command Line if you want to know how to become a command line power user.

Back to today’s topic. You’re starting to learn how to use the command line, but it’s annoying to always have to type in similar commands over and over. If only there were a way to pull up commands that you’ve already run to run again as is or to quickly modify. Today, I’m going to teach you how to do exactly this.

Categories Linux, Tips 'n Tricks
Comments (23)

Convert DOS-Formatted Files to Unix-Format in Ubuntu and CentOS

by Chris Jean

March 8th, 2009

Have you ever seen a bunch of ^M characters in a text file? This odd character at the end of a line can also be represented as a Ctrl+M or <CTRL>M. You don’t know what it is, and you want it to go away.

Today, I’ll help you understand what that odd ^M character is, why it is in some of your documents, and how to get rid of them.

Categories Development, Linux, Tips 'n Tricks
Comments (4)

Multitasking from the Linux Command Line + Process Prioritization

by Chris Jean

March 6th, 2009

Did you know that you aren’t limited to working on one thing at a time while on a Linux command line? You can actually “minimize” a program that you are in, get back to the command line, and then return to the program whenever you’d like.

When you run a program or script on the Linux command line (from now on referred to as the shell), you are creating a new job. For those that are used to GUI environments, each of these jobs is somewhat like a window on the desktop. Just as you can have multiple windows and switch between them, the shell is capable of managing multiple jobs and allows you to switch between them.

There is a lot to cover, so let’s start simple by describing what states a shell job can be in.

Categories Development, Linux, Tips 'n Tricks
Comments (39)

4 Great Tools to Find Files Quickly in Ubuntu

by Chris Jean

March 4th, 2009

Many of you fellow Ubuntu users will be familiar with the “Search for Files” tool that allows you to look for files. As is true with most things in Linux, there are great desktop tools, but more power can be found in Terminal than any streamlined desktop tool can match.

Today, I’d like to introduce you to a few tools that can turn a chore of finding files into an easy process.

locate

The first tool that you should become comfortable with is also one of the simplest. The locate program works in a similar fashion to most graphical search tools.

Using

Let’s say that you’ve read up on how to modify X to provide enhanced video performance. The document that you are reading says to modify the xorg.conf file, but the document doesn’t say where that is and you don’t know how to find it. You can use locate to find it for you quickly:

[chris@home ~]$ locate xorg.conf
/etc/X11/xorg.conf
/usr/share/man/man5/xorg.conf.5.gz
/var/lib/x11/xorg.conf.md5sum
/var/lib/x11/xorg.conf.roster

It’s clear that /etc/X11/xorg.conf is the file we’re looking for, but what if dozens or even hundreds of results were returned? How could you manage all of that? If you know a little bit about regular expressions (every Linux user really should learn the basics), you can easily use regular expressions to search for directories or files. For example, we can search for an exact match for xorg.conf with the following command:

[chris@home ~]$ locate -r '/xorg.conf$'
/etc/X11/xorg.conf

If you want to look for files without case sensitivity, you can use the -i option. For example:

[chris@home ~]$ locate -i 'monthly report'
/home/chris/Documents/August Monthly Report.doc
/home/chris/Documents/July Monthly Report.doc
/home/chris/Downloads/latest monthly report.doc

Something that may confuse you when you first start using locate is that it not only returns matches, but if the match is a directory, it also lists all of the directories and files under that directory. For example:

[chris@home ~]$ locate Desktop
/home/chris/Desktop
/home/chris/Desktop/Head Shot.jpg
/home/chris/Desktop/Skype.lnk
/home/chris/Desktop/Steam.lnk
...

That command returns more than a thousand entries on my system. To reduce the results down to a list of entries that match the search term without all the contained files and directories, use the -b option:

[chris@home ~]$ locate -b Desktop
/home/chris/Desktop
/home/chris/bin/recordSoundAndDesktop.sh
/opt/HipChat4/lib/QtQuick/Controls/Styles/Desktop
/usr/include/qt4/QtGui/QDesktopServices
/usr/include/qt4/QtGui/QDesktopWidget
/usr/lib/libreoffice/program/wizards/common/Desktop.py
/usr/lib/libreoffice/share/gallery/computers/Computer-Desktop.png
/usr/lib/python2.7/dist-packages/xdg/DesktopEntry.py
/usr/lib/python2.7/dist-packages/xdg/DesktopEntry.pyc
/usr/lib/python3/dist-packages/xdg/DesktopEntry.py
/usr/lib/python3/dist-packages/xdg/__pycache__/DesktopEntry.cpython-35.pyc
/usr/lib/x86_64-linux-gnu/qt5/qml/QtQuick/Controls/Styles/Desktop
/usr/share/kde4/apps/kio_desktop/DesktopLinks
/usr/share/man/man3/File::DesktopEntry.3pm.gz
/usr/share/perl5/File/DesktopEntry.pm
/usr/share/sounds/KDE-Window-All-Desktops-Not.ogg
/usr/share/sounds/KDE-Window-All-Desktops.ogg

There are still quite a few matches, but we no longer have a sort through all the extra results to find what we’re looking for.

Important Note

The locate command relies on a slocate database to function. This database caches the file system directory and file names and allows the locate command to operate in moments rather than taking long periods of time to search the entire file system.

This ability doesn’t come without cost though. By default, your system will run a command to update this database once a day at 6:25am. This command will update the database with any changes made to the file system since the last update.

If you’ve made changes recently, such as added or removed large amounts of files and directories, and you want to use the locate command to search through these new directories and files, you can manually update the slocate database at any time with the following command:

[chris@home ~]$ sudo updatedb

Depending on the volume of changes made since the slocate database was last updated, this could take a few seconds to a few minutes to update. Once it’s done, you’re ready to search again with locate.

which

locate is great for searching the entire file system. However, sometimes you just want to find where a command lives. That’s where which comes into play.

which only does one thing, but it does it well. Given the name of a command, which will tell you where the file for that command resides in the file system.

Let’s imagine that we want to find out where the firefox command resides. We can simply run the following to find out:

[chris@home ~]$ which firefox
/usr/bin/firefox

Why would you ever want to search specifically for a command? Imagine that you have some code that needs to replace an old command. You really need to remove the the old one (I usually rename it to the same name followed by “.bak” to keep it around in case I need it) first before putting the new one in its place. which will help you find the file quickly and easily so you don’t have to search everywhere to find it.

whereis

The whereis command is similar to which but goes one step further. Not only does it tell you where the executable file is located at, but it will also locate the source, man page, and other associated directories as well.

Keeping with the which example, let’s ask whereis about the firefox command:

[chris@home ~]$ whereis firefox
firefox: /usr/bin/firefox /usr/lib/firefox /etc/firefox /usr/share/man/man1/firefox.1.gz

This helped me find /usr/lib/firefox/plugins earlier when working on the solution for Flash as that location is where the global add-ons for Firefox are stored.

find

find is the ultimate search tool, the Swiss Army knife of Linux search if you will. find can do everything from search for files based on last modified times, owners and groups, permissions, type (file, directory, symbolic link, etc), whether the file is readable/writable/executable by the current user, file size, whether the file or directory is empty, and much more. You can also search for matches by regular expression.

When you use this tool in combination with the xargs command (which I’ll have to cover some other time), you can do truly amazing things on the command line. Imagine being able to recursively search through the current directory for files ending with “.bak” that were last modified more than 100 days ago and then deleting them with one command:

[chris@home ~]$ find . -name '*.bak' -type f -mtime +100 -print0 | xargs -0 /bin/rm -f

It will look very complex at first, but when you master the art of using find, it will become second nature. Soon, you’ll wonder how you ever searched for things without it.

To be frank, there is much more to find than I am willing to cover and there are many, many great introductions to the tool that do a better job than I would. So, I’ll simply point you to three great references that can be used to get going with find.

Example uses of the Linux Command find
(I apologize for the horrible colors in advance) Unix/Linux “find” Command Tutorial
Using the find command

Happy searching.

Categories Linux, Tips 'n Tricks
Comments (9)

Change Timezone in CentOS

by Chris Jean

March 3rd, 2009

You just got your new CentOS dedicated server, and you notice that times in your logs aren’t quite right. You check the time from the command line (run date), and find that the timezone is set to US Eastern or some other timezone. How do you get this changed?

Unfortunately, this is not an easy thing to figure out. Fortunately though, it’s not hard to do with the right directions.

Categories Linux, Tips 'n Tricks
Comments (85)

Using Vi/Vim as a Command Line Editor

by Chris Jean

February 27th, 2009

In many of my Tips ‘n Tricks or tutorial posts, I have commands that involve editing/creating files in Vi. For those that have never used Vi, it can be a daunting experience trying Vi for the first time.

There are many holy wars fought over the best editor to use from the *nix command line. I won’t claim that Vi is the best; rather, Vi is simply what I’m used to, it’s powerful, and it is always available on servers that I work on.

In order to best help those that don’t know how to use Vi but are interested in learning, this will serve as a very brief tutorial on how to get started.

Categories Development, Linux, Tips 'n Tricks
Comments (4)

Unzip Multiple Files from Linux Command Line

by Chris Jean

February 26th, 2009

Here’s a quick tip that will help you work with multiple zip files on the command line.

If you are in a folder and have three zip files in it (a.zip, b.zip, c.zip) that you want to unzip, “no problem,” you think. “I can take care of that with one command.” And you quickly run the following:

[gaarai@tenshi ~]$ unzip *.zip

However, rather than having the unzip program nicely unzip each file one after another, you receive the following:

[gaarai@tenshi ~]$ unzip *.zip
Archive:  a.zip
caution: filename not matched:  b.zip
caution: filename not matched:  c.zip
[gaarai@tenshi ~]$

I’m sure that this is not what you were expecting. I know I certainly wasn’t expecting this when I first tried it. However, this problem can help us understand more of how the command line works.

The Problem

Whenever you use a wildcard (*), the shell itself will expand that and pass the results to the program rather than the program handling the expansion itself. That means that our previous command was actually expanded to the following before being executed:

[gaarai@tenshi ~]$ unzip a.zip b.zip c.zip

Again, this may not look odd since other programs (such as mkdir, chmod, etc) can take one or more arguments and repeat the process for each. However, unzip is different and actually has use for the additional arguments. If you specify additional arguments after the zip file name, unzip will try to only extract those specific files from the archive rather than all the files.

The previous command told unzip that it should extract b.zip and c.zip from inside the a.zip archive. Since those files don’t exist inside a.zip, we get the unexpected error output seen earlier.

You might think that you will have to manually unzip each archive one at a time, but you’d be wrong.

The Solution

Just because the shell expands out wildcard characters automatically doesn’t mean that programs can’t as well. The simple solution to this problem is to quote the argument to prevent the shell from interpreting it:

[gaarai@tenshi ~]$ unzip '*.zip'
Archive:  c.zip
   creating: c/
  inflating: c/test.txt

Archive:  a.zip
   creating: a/
  inflating: a/test.txt

Archive:  b.zip
   creating: b/
  inflating: b/test.txt

3 archives were successfully processed.
[gaarai@tenshi ~]$

That’s all there is to it. I’ve seen solutions from coding loops in bash script to running find combined with xargs, but none of that is necessary. Simply let the unzip command itself take care of things for you.

Categories Linux, Tips 'n Tricks
Comments (36)

SSH Tutorial for Ubuntu Linux

by Chris Jean

February 19th, 2009

I touched on this topic in my First Day with Ubuntu at the Office post under Accessing Remove File Systems; however, I thought that this deserved its own topic.

For those that don’t know SSH stands for Secure SHell. In very non-technical terms, SSH allows you to access a system running a SSH server over a network. This other system can be another computer in your home or a remote system on the other side of the planet. SSH will allow you to connect to that other system and communicate with it securely. All the data passed back and forth is encrypted, so you don’t have to worry about people sniffing your passwords or valuable data.

You may be asking what all of that means to you. What it means is that you can connect to and control a computer that is somewhere else with the computer that is sitting right in front of you. If you have a remote web server running Linux, SSH will let you install software, edit files, change the server configuration, access the database, and more. Did you forget a file on your computer at home? No problem, just SSH into your home computer and send the file to your email account or copy it directly to your office computer.

Hopefully those quick examples of what you can do is enough to whet your appetite.

Categories Development, Linux, Tips 'n Tricks
Comments (11)