All of the WordPress themes that I work on for iThemes are managed as Git repositories. Recently, we moved past the 100 repositories mark. That’s a lot of repositories to manage, and unfortunately, too many of those repositories contain duplicated information.

Later on, I might delve into how we use Git to manage our theme repos. For today, however, I’d like to focus on how I quickly and easily pushed up changes to more than a dozen repos in a single, albeit long, Bash command.

I had finished making updates to 16 Flexx repos, and I needed to push all of those changes up. Since I had multiple working repos in that folder, I was lucky that each of these repos began with the text “Flexx”. Also, since they are all part of the same series and need to keep the same version number, that simplified the tagging as all could be tagged as 2.5.0.

Given this information, I simply ran the following command from the directory that contained all the repository directories:

There’s a lot going on here, so I’ll break it up and explain what I’m doing.

This is basically a compound command. Bash’s for command is being used to step through the results of ls|grep Flexx. Each result will be stored to the variable i which can be referred to with $i.

In other words, I’m going to loop through each directory that contains the text “Flexx” (technically, it isn’t limited to just directories, but I don’t have any files containing “Flexx”, so I’m safe). For each iteration of the loop, the variable $i will contain the name of the folder I’m interested in.

The most important bit here is do. do simply begins the functional part of the loop and could be followed by any command that I wanted to start the loop with.

The echo command isn’t technically necessary. I simply have it there so that I can better determine what output belongs to which repository.

This is a very important command. It changes the directory to the repository I wish to run commands on.

This is the command that actually does what I want to do in each repo directory. This adds all the modified files, commits them with a message of “2.5.0″, pushes the changes to the remote repository, creates a new tag of 2.5.0, and pushes up the new tag.

I have all of these commands chained together with && so that if one command fails, the other ones won’t run. This prevents tagging in the event that the repository couldn’t be committed or pushed.

It’s a small command but necessary. This switches back to the main directory that holds all the repos, thus returning us back to a state that the next loop iteration can work with.

As with the other echo command, this simply allows me to keep track of what is going on more easily.

The done command finishes out the loop iteration.

I hope that by sharing this you can gain a bit of insight into how I work with repositories while also learning more about how you can do things with Bash.

Now that I know how to do this, it seems so easy and straight-forward. To change the default application files of a specific type are opened with, do the following:

1. Right-click a file that you wish to change the default application for and select Properties.
2. Click the “Open With” tab.
3. Select the desired application’s radio button.
   • Additional applications can be added if the one you want is not listed. Use the Add button to find the desired application.
   • You can also remove applications from the list by highlighting the application and clicking the Remove button.
   • Adding applications to or removing applications from this list changes which applications are available in the “Open With” option when you right-click a file.
4. Click the Close button.

Again, it seems so easy and straight-forward now. Go figure.

It’s been a month since my last post. For all my regular readers, I’m very sorry for the absense.

There’s a lot of intersting stuff going on right now. Fortunately, with so much going on, I shouldn’t have a lack of topics to talk about.

To get the old post ball rolling again, how could I not start back up with Ubuntu 9.04?

Ubuntu 9.04, Jaunty Jackalope, is scheduled to be released on April 23rd. Even though it has been a mere six months since Intrepid Ibex went to release, the Jackalope promises some good stuff:

• GNOME 2.26 -  This latest release of GNOME includes a number of features, such as: better multi-monitor support (I look forward to testing this out), a new tool to handle burning disks (Brasero), a plugin promising easier file sharing (even on Bluetooth connections), UPnP media playback support in the GNOME Media Player, new volume control that supports per-application volume control when using PulseAudio (I love this feature in Vista and am excited to get it in Ubuntu). This is just a few of the features that picqued my interest. For more details, see GNOME 2.26′s release notes.
• X.Org Server 1.6 – This new version of the X server promises great improvements to the free drivers available for a large variety of cards, primarily for Intel and AMD graphics. The AMD deal is very exciting. It’s a big topic though, so I’m going to talk about it in more detail later.
• Linux Kernel 2.6.28 – As with most kernel releases, a lot of things have changed. Some of the changes that I found interesting (and that I could understand the value of) are: stable implementation of the Ext4 file system and improved memory management for GPU memory via GEM (the foundation for improving Linux graphics in the future). For a full list of features, check out the breakdown at Kernel Newbies.
• Eucalyptus – Adds cloud computing capabilities to Ubuntu’s server offering. I’m excited to try this out.
• Check out the release notes overview for more information about the new features.

As with most new software, there are some caveats to be alert of. I recommend reading the release notes if you are upgrading. Pay specific attention to the removal of Ctrl+Alt+Backspace support by default and the change to update notifications.

I hope that you are looking forward to the release as much as I am. When the Jackalope goes golden and I have a chance to test it out, I’ll definitely post about my experiences here.

Linux has many great tools built in that help maintain the system without user intervention. One such tool is Cron.

On my Ubuntu 8.10 system, there are many things that are set to run each day: locate database updates, misc cleanup utilities, automatic package updates, log rotations, etc. All of these are managed by the Cron system.

For a while, I needed to manually run the updatedb command to update the locate database, and I never thought about why. The problem is that my daily, weekly, and monthly Cron jobs never run. The reason for this is that these Cron jobs are scheduled to run very early in the morning, when my system is off. Thus, these job schedules never run.

The solution for this is easy. I simply need to change the times these run at to times when my system is on.

Some of you may be unfamiliar with Cron and how to set up jobs for Cron to execute, so let’s look at my /etc/crontab file quickly to see what the problem is.

The four lines at the bottom are the parts of interest. This looks like a jumbled mess, but it’s easy to understand once you break it down section piece by piece. Each part is as follows:

• Minute the command is to be run at
• Hour the command is to be run at
• Day of the month the command is to be run at
• Month the command is to be run at
• Day of the week the command is to be run at
• User to run the command as
• The actual command to run

Given this information and the fact that an asterisk (*) means all, the last line means: at 6:52am on the first day of each month, have the root user run “test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.monthly )”.

So, the four lines at the bottom of my /etc/crontab file have my system run all the commands in /etc/cron.hourly every hour of every day at 17 minutes past the hour, all the commands in /etc/cron.daily every day at 6:25am, all the commands in /etc/cron.weekly on the 7th day of each week at 6:47am, and all the commands in /etc/cron.monthly on the first day of each month at 6:52am, respectively.

Since the specific Ubuntu machine that I am talking about is my office machine and I’m rarely in the office before 7am, these Cron jobs don’t get a chance to run like they normally should. It should be noted that they run at the specific times they do so that users who start to access the machines at regular school or office times will have a system that very recently ran these commands, providing fresh logs, a recently updated locate database, etc without bogging the system down as they begin to access it.

I want to change the time to something later but not so late that I don’t get to benefit from the commands until late in the day. The best time that meets these requirements for my use is changing 6am to 8am for each of the last three commands.

The following is my modified /etc/crontab file:

Notice thatall I had to do is change the last three lines’ second number from a 6 to an 8. All I have to do now is save the file, close it, and wait until tomorrow for the commands to be run starting at 8:25am.

If you want to modify your /etc/crontab file, keep in mind that you will need root access to do this.

You can open the file in Vi using:

If you are more familiar with graphical rather than command line editors, you can open up the file for editing in Gedit with the following command:

Now you can enjoy making use of Cron even if you don’t have your Ubuntu machine on 24/7.

There’s a very good chance that you have already heard of the game World of Goo. If not, check out this video to get a taste.

This game has a number of great things going for it:

1. It’s a heck of a lot of fun to play.
2. The world if very unique and has a fun feel to it.
3. The music really sets the mood for the different areas.
4. It supports Windows, Mac, and Linux.
5. It’s DRM free.

These days, I find great favor in things that 1) have Linux support and 2) are DRM free. Since World of Goo looked like a ton of fun and had both of those, the $20 price tag was nothing. Frankly, the price is very small when you consider the many hours of fun that you can have with all the goo balls.

I ran World of Goo both on one of my Windows Vista machines and on my Ubuntu machine at home. It worked very well on both platforms. Even the Linux version was very smooth even though I’m running Compiz.

FYI: If you want to run this on Linux, you can get the software as a DEB or RPM package or as a tar.gz archive with all the application files. I recommend usingone of the package files. Since the packages are built for 32-bit, you will need to use these instructions if you have a 64-bit distro.

I highly recommend World of Goo, but you don’t have to have blind trust in my opinion. You can download the demo and try it out for yourself.

You might be interested in their post about the release of the Linux version. The day of the Linux release, sales were 40% higher than their previous highest-selling day. “There is a market for Linux games after all ,” said one of the developers after updating the post.

I recently watched the entirety of the Revolution OS documentary. While it definitely is not a video that can be enjoyed by most people, nor even most computer users, it is a very intersting watch for a number of reasons.

Visiting the documentary’s site, you quickly gain a sense of what this documentary is aimed to be: “… the inside story of the hackers who rebelled against the proprietary software model and Microsoft to create GNU/Linux and the Open Source movement.” However, that’s not what this video is about.

When I first started watching the documentary, I thought I would get this intimate insight into how people made calculated movements against the growing monopoly of Microsoft’s operating systems. In fact, this isn’t anything near what actually happened.

Microsoft and its actions were not the focus of anyone involved in getting the ball rolling on the open source movement. Richard Stallman was motivated to create the Free Software Foundation and started building GNU not because of Microsoft, but because of the actions of administrators that wanted to control access to computer systems, by hardware vendors that didn’t release the source for their drivers, and by a myriad of other things. If Bill Gates and his company, Micro-Soft (as it was called at the time), was any consideration, it was merely another straw on the camel’s back.

There are many other surprises like this in the documentary, and it is why I highly recommend those that are interested watch it and pay attention. You will gain a good beginning understanding of the foundation and history of GNU, GPL, FSF, Linus Torvalds and the Linux kernel, how the Linux kernel created the opportunity for today’s Linux distros, how the open source movement began, how the open source business model works, and much more.

The documentary may not be perfect. The creators may not have understood what they actually produced. And the material is actually quite dated by now. Just the same, it’s a very good watch, and I recommend that you check it out.

A few days ago, I blogged about changing the timezone on a Linux server. In the post, I mentioned how the zoneinfo files needed to be updated in 2007 due to congress expanding the number of days that Daylight Saving Time covers. However, I did not go in depth about how to update the zoneinfo files.

Since then, I’ve received many search queries from people looking for information about why their server did not properly update when the Daylight Saving Time change hit. For instance, I got queries of “is daylight savings default in centos”, “daylight savings time didn’t change centos”, “dst timezone change centos”, and many more. It’s clear that there are server administrators out there that are very confused about the Daylight Saving Time situation on their server.

Today, I hope to remedy that problem and give server administrators the information they need to update these zoneinfo files. I typically focus in on specific platforms, but today, I’ll try to cover as many distros as I can as well as provide a universal solution.

DST Update, Linux, and Your Server

The United States Congress signed the Daylight Saving Time change into law in 2005. In this new law, Daylight Saving Time starts on the second Sunday in March and ends on the first Sunday in November. The change took effect in the US in 2007 and will be the standard until the US Congress modifies it again.

In 2006, new zoneinfo files for Linux were released that had the new calculations necessary to account for this change in US timezone information. Most systems should have been updated automatically, but if your system doesn’t do automatic updates, the update system is malfunctioning, or the updated information wasn’t made available for your system, you’re left with a server that has its time out of sync for many days out of the year.

Checking for Old DST Calculations

A good way to find out if your zoneinfo files are updated with the new calculations is to use the time zone dumper, zdump. For example, I’ll ask my local machine running Ubuntu when the pertinent DST time changes are in 2009:

Note: I removed some of the output to make it fit.

It really doesn’t matter which timezone is used since all the US timezones will report the same changes. I simply used Central since it’s my timezone.

Notice how it shows that the CST to CDT switchover point is at 2am on March 8 and how the CDT to CST switchover point is at 2am on November. This is what we want to see as it indicates that my system is using the new DST calculations and not the old ones.

So, if your system does not respond with similar output and you are checking a US timezone that includes DST calculations, you need to update your zoneinfo files.

As an odd aside, it’s interesting to note that March 8th is the earliest possible second Sunday in March and November 1st is the earliest possible first Sunday in November. We get to see this interesting arrangement again in 2015, 2020, and 2026 (more years follow of course, but I won’t bore you with them). This has no importance to the topic; I simply like patterns.

Updating zoneinfo Files the Easy Way

Fortunately, upading your zoneinfo files should not be difficult if you are using a fairly modern distro that has a package manager. The tzdata package is available on all the distro repositories that I tested and is the package we want to install/update to fix up the zoneinfo files.

Note: In order to update zoneinfo files, you must have root access. If you do not have root access, contact your hosting provider or your sysadmin and provide them with a link to this topic with a request to update the files.

For each package management system, the same steps will be followed. First, install the tzdata package. If the package manager indicates that the tzdata package is already installed, update the package. If both of these methods fail, skip down to the next section to update the files manually.

Note: You need root access to run the following commands, either access a root shell or prefix the command with ‘sudo ‘ in order to run the commands successfully.

yum

yum is the standard package manager used by most RPM-based distros, such as: CentOS, Fedora, and Red Hat Enterprise Linux (RHEL).

Install: yum install tzdata

Update: yum update tzdata

APT

APT is used by Debian and its derivitives, such as Ubuntu, Linux Mint, MEPIS, Damn Small Linux, and many others.

Install: apt-get install tzdata

Update: apt-get update tzdata

Updating zoneinfo Files Manually

If you were unable to install/update the tzdata package, were unable to find a tzdata package, or don’t have a functional package manager, you can still update your zoneinfo files. It will take a bit more work however.

The following commands will retrieve the new data, prepare it, and install it to the /usr/share/zoneinfo directory.

We now need to access a root shell. If you can use sudo, you can also just add “sudo ” to the beginning of the following command in order to temporarily gain root privileges.

If you are on an old system, you may get the following error:

This means that your zoneinfo files are probably located at /usr/lib/zoneinfo. If this is the case, run the following command:

Now it’s time to test the changes.

Testing Changes

Like before, confirm that the proper dates and times are found in your zoneinfo files for the 2009 DST changes:

Note: I removed some of the output to make it fit.

If you don’t receive the appropriate output, please leave me a comment with the distro you are running and what steps you have tried.

Now we just need to check that the system has the correct time. Simply run “date” from the command line and verify that the correct date and time are reported. If the time is still incorrect, you need to update your /etc/localtime file. You can find instructions on how to do that here.

I’ve had a lot of fun recently posting about how to do stuff on the command line in Linux. My focus is specifically for Ubuntu users, but the information and techniques can be used for any Linux distro.

Since I’m probably going to end up with a lot of content under this topic, I’ve decided to create a dedicated tag: Mastering the Command Line. I’ve gone through my older posts on this topic and tagged them as well. So, make sure to check out Mastering the Command Line if you want to know how to become a command line power user.

Back to today’s topic. You’re starting to learn how to use the command line, but it’s annoying to always have to type in similar commands over and over. If only there were a way to pull up commands that you’ve already run to run again as is or to quickly modify. Today, I’m going to teach you how to do exactly this.

Introduction to BASH Command Line History

BASH (Bourne Again SHell) is the default command line shell that is used in Ubuntu, and most other distros for that matter. The instructions that I give here are specific to BASH and may or may not work in other shells. Since most people don’t modify which shell they access through Terminal, I only point this out to inform those who have changed their shell that their experiences may differ depending on the shell used.

When you close a Terminal session, BASH writes all the commands you’ve run for that session to your ~/.bash_history file. When you load a new session, this file is read and those commands along with the new commands you execute for that session become your new command history.

It’s important to keep in mind that commands executed for a session are only written when you exit. If you have two sessions running concurrently, they cannot cross-reference the newly-executed commands run in the other session. Furthermore, if you force a Terminal session to quit, the command history will not be updated.

Simple Command History Navigation

The easy way to get started with your command history is to simply navigate with the up and down arrow keys. Using the up key, you can scroll through previous commands one command at a time. To scroll back down the list, you can use the down arrow key.

Once you’ve found the desired command, you can treat it as if you’ve typed that command in the command line. You can simply press Enter to execute the command again, or you can modify the command as needed before executing it.

This method is very good for quickly running commands that you have to execute repeatedly without many other commands between.

Listing and Searching Command History

BASH provides the history command that when executed with no options will list the entire command history. Each command will be preceeded with a number that represents that command’s command number. I’ll tell you how to use these numbers below.

You can also have the history command produce just the last certain number of commands by supplying a number. For example, if you want to see the previous ten commands, you can run the following:

Notice that the last command is the same as the command you ran to produce it. So, “history 1″ will just show you “[command number] history 1″.

Searching History

If you remember that you ran a really cool command and can’t quite remember how you did it, combining history with grep provides an easy way to find that command with just a piece of the command.

For example, let’s say that I ran a command that listed all the files that end in “.php” inside the ~/wordpress folder it then filters for just matches that contain the text “link” and then reverses the output. I was quite proud of that command, but I can’t remember how I did it. So, I search through my history.

Since I don’t think that I have the word “link” in many of my commands, I pick it as the target to search for and run the following:

That’s what I was looking for. Now I can bask in the glory of my ingenious command and use it as the inspiration for other inspired words of command line magic.

Running Previous Commands Again

Searching through the history is great and all, but I’d rather not have to copy and paste found commands. I’d like to just run them immediately. BASH to the rescue again.

If you’d like to run the previous command again quickly, you can simply run “!!”. The first exclamation point tells BASH that you are running a previous commands. The second exclamation point tells BASH that it’s the last command that you’d like to run again. It’s a double exclamation point simply to make it quick and easy to type in.

I’ve already shown you how to look through your history and find a specific command in your history complete with the command number. Now is the time to make use of that number. If I wanted to execute command number 510 (which bash) again using the history, I can simply run the following:

Notice how running the command again first lists the actual command and then the output.

BASH substitutes the !number command with the actual command from the history. This allows you to supply additional parameters to the command. The following is a silly example, but it gets the point across.

Notice how the command now has the additional parameter added to the original command. As expected, the full command is listed followed by the command output.

You can also do the standard manipulations such as piping the output, “!412 | grep wp-admin”, or redirecting the output, “!510 > shell_locations.txt”.

Searching and Running Previous Commands

The “!” functionality of BASH also allows you to search through the history and execute the first match.

The basic form of this simply searches for matches at the beginning of the command. “!find” executes the most recent command that begins with “find”.

You can also search the entire command for a match. “!?link?” executes the most recent command that contains “link”.

Advanced History with Regular Expressions

As with many tools in the Linux world, you can attain great power by combining the tool with regular expressions. The history in BASH allows you to run previous commands with a search and replace in Regex fashion.

Let’s take my searching through the ~/wordpress example, command number 412, and run it again with some modifications. This time, instead of searching ~/wordpress, I’d like to search through /home/site/public_hmtl/wordpress. I’d do that with the following:

Note that the search and replace is done by adding “:s|find|replace|” to the end of the history command. I used the pipes, “|”, to deliminate the find and replace portions to make it easier to add the forward slashes, “/”, to the terms. You can also use “:s/find/replace/”.

Closing Thoughts

I hope that you enjoy my Mastering the Command Line series. If you have any command line-specific requests, please leave a comment, and I will see about creating a tutorial on that topic.

Remember, those who forget history, are doomed to repeat it by manual entry.

Have you ever seen a bunch of ^M characters in a text file? This odd character at the end of a line can also be represented as a Ctrl+M or <CTRL>M. You don’t know what it is, and you want it to go away.

Today, I’ll help you understand what that odd ^M character is, why it is in some of your documents, and how to get rid of them.

All About the Newline

In text documents, lines are separated by what is called a newline (also known as a line break or end of line). Different operating systems have different character codes that represent this newline.

For example:

• DOS-based systems, including Windows, as well as a number of other, older non-Unix OSes use a carriage return (CR) character followed by a line feed (LF) character.
• Commodore and the old Apple OSes before OS X used a CR character. Since OS X+ is based partly on BSD, which is in turn based on Unix, the new Apple OSes use the Unix newline method described below.
• Unix and its derivatives (Linux, BSD, and others) all use the single LF character to represent a newline.

So what does all of this mean to you? It means that text documents that come from a Windows system won’t always play nice in Linux. The converse is true however. If you create a text file in Linux, many programs will fail to recognize the single LF as a newline and will render the document without any line breaks.

dos2unix and unix2dos to the Rescue

Fortunately, there are a couple of very easy to use programs that make dealing with this file format mess much easier. They are dos2unix and unix2dos.

These programs basically do exactly what their name implies: dos2unix takes a file and converts all DOS-style newlines to Unix-style newlines. unix2dos takes a file and converts all Unix-style newlines to DOS-stlye newlines.

I put Ubuntu and CentOS in the title because I’m going to give instructions for installing these programs on each of these distros. Why just these two? They are the two that I work with most often and are representative of the lion’s share of what people are using these days. If you need help with a different distro, please let me know in a comment.

Installing dos2unix and unix2dos in Ubuntu

There aren’t any dos2unix or unix2dos packages that can be found in Synaptic; howver, there is a packages that will install them for you. Simply open up Synaptic and install the tofrodos package. If you are like me and prefer to do this from the command line, you can run the following command:

That’s all there is to it. Not only will dos2unix and unix2dos install, but alias programs fromdos and todos will be installed as well. These additional programs work in the same manner, so it’s purely a matter of preference which ones you use.

Installing dos2unix and unix2dos in CentOS

I really thought that I had to install these in CentOS, but amazingly, the programs are already installed by default. I tested this in a Virtual Machine fresh install, and the programs were there on the first boot.

So, CentOS users, you’re already good to go.

Using dos2unix

Fortunately, using these programs couldn’t be easier.

Let’s say that you are in a Terminal (Applications > Accessories > Terminal) viewing text files. Maybe you just downloaded a new WordPress plugin and you are reading the readme.txt file. It doesn’t really matter. However, there is a problem. The readme.txt file has a bunch of ^M characters at the end of each line, and it’s really distracting.

Simply exit out of the editor you are currently in, since the file will be modified, and run the following command:

If there are multiple files, you can specify each one with a space separating each. For example:

The program doesn’t produce any output. Simply reopen your text file and look at all the beautiful non-existant ^M’s.

Using unix2dos

The unix2dos program has the exact same syntax as dos2unix. However, I thought it might be helpful to describe a situation in which you might need to use it.

You’re working on a project. You’ve just sent out your batch of files, and another member on the project complains that you are being a jerk and remove all the newlines in your text files. This other member is most likely using a Windows application that doesn’t understand the Unix newline format. Rather than getting into a format war, it’s typically better and quicker to simply convert your text files to a DOS/Windows format.

If you have a folder full of files that all need to be converted, simply run:

Now you can send these new files to your project group and hopefully avoid any more unproductive drama.

File Formats in Vi

If you happen to use Vi, you can change Vi back and forth between DOS and Unix modes for newlines with a simple command. “:set ff=dos” sets the editor to use DOS newline encoding and will save the file in a DOS-encoded format. “:set ff=unix” sets the editor use Unix newlines and will save the file in a Unix format.

Note that changing the format to dos from unix will always work as expected. This is because any file that contains just LF characters for new lines will be converted to CRLF while lines that already end in CRLF will be left as is.

If your Vi config defaults to a unix format and you open a DOS file, you will see the ^M characters. You can either use the dos2unix conversion utility first or change Vi first to the dos format and then to unix.

If you’d like Vi to default to DOS or Unix formating each time you start a new Vi session, you can add the setting to your ~/.vimrc file. In that file, either add “set ff=dos” or “set ff=unix” depending on your needs. Note the lack of the colon, :, in the .vimrc entries.

For more information on the ff or fileformat setting in Vi, check out the official documentation.

Closing Thoughts

Maybe one day we won’t have to worry about these types of things. For now however, it’s good to know the tools that make these problems easily manageable.

BTW, Happy Daylight Savings Day everyone. I hope you enjoyed the loss of an hour of sleep.

I’m more or less taking the night off. I’ll leave you some good reading material that should cover just about any reader that comes here.

• Make Linux: Harder – Better – Faster
  This page is a great compilation of links on how to improve security, speed, functionality, or appearance of your Linux system.
• Firefox Minefield
  Very interesting version of Firefox that is in development. Apparently, it is capable of massive speed and gives Chrome a run for its money.
• Conky
  A highly-customizable graphical system monitor for your Linux desktop.
• Armed and Dangerous
  A blog I just happened upon today that has many great reads. The topics are varied, but there should be a little something for everyone. I was hooked with the My comment to the FCC on DRM post, as I share many of the feelings that the author does on the subject.
• Swift Weasel
  This project builds Firefox from source to provide optimized builds for the Linux platform. It’s still in the early stages but has potential.

I hope that my big post yesterday makes up for my weak one today.