How to Create a Cron Job (Scheduled Task) for Your Website or Blog

How to Create a Cron Job (Scheduled Task) for Your Website or Blog

by Christopher Heng, thesitewizard.com

On occasion, you might come across the need to create a scheduled task for your site. For example, you may have installed a website statistics software such as Awstats or a content management system such as Drupal that requires a background program to run at a certain time. In such a case, the software's documentation often asks you to schedule a cron job on your web server. This article provides a step-by-step tutorial on how you can schedule such a task using a program named crontab.

System Requirements

1. An Appropriate Operating System

   Cron jobs, created using the command line program called crontab, require that your website be hosted on a Unix-type web server, such as Linux or one of the BSDs. Generally, if the web server that your site is on is running Windows, you cannot use this tutorial. Note that I am referring to the computer which your web host uses to place your site, not your own computer.

2. Shell Access or Control Panels Interface to Crontab

   You also need to be able to connect to your web host using telnet or SSH. Alternatively, your web host must provide you a way to set the crontab tasks using their control panel.

   This tutorial assumes that you have shell access, which means that you can connect to your web server using SSH or telnet and work from the command line on that computer. However, even if you can only set cronjobs from your web host's control panel, once you learn the basics of the crontab commands, you should be able to easily transfer that knowledge to using the control panel. As such, this tutorial should still be useful to you, since even the most user-friendly control panel I've encountered is usually quite cryptic about how you can set up a cron job.

   Note: if you use a free web host, you probably do not have shell access. Neither do most of them allow you to set cron jobs even if their control panel has that facility. Generally, if you need to set a cron job, chances are that you will need a commercial web host.

Overview

Basically, we will start by creating a schedule. Then, we will feed this schedule to a program called crontab. Crontab will take the schedule and install it into an internal set of tables which it manages. At the appropriate time demanded by your schedule, another program called cron will execute the tasks you have set.

Steps to Setting Up a Cronjob on Your Web Server

1. Figure out the schedule

   The first thing you need to do is to figure out a schedule for your task. Do you want it run once a day? Hourly? Note that if you use a shared web server, where many websites reside on the same web servers, you should not run your cronjob too frequently, or you will affect the performance of both your website and the other sites hosted on the same computer (and get you booted off the web host as well). If you really need such a high frequency, you may need to get a virtual private server or even a dedicated server where you have the machine to yourself.

2. Figure out how to write the schedule for crontab

   The next thing you need to do is to write your schedule in a way so that crontab will understand it. The crontab format is somewhat arcane and cryptic, so we will take this step by step.

   The basic format of a crontab schedule consists of 6 fields, placed on a single line and separated by spaces, formatted as follows:

   minute hour day month day-of-week command-line-to-execute

   The acceptable values for each of the 6 fields are:

   Field	Range of values
   minute	0-59
   hour	0-23
   day	1-31
   month	1-12
   day-of-week	0-7 (where both 0 and 7 mean Sun, 1 = Mon, 2 = Tue, etc)
   command-line-to-execute	the command to run along with the parameters to that command if any

   The fields have to be in that exact order, with no empty or missing fields, and everything must be placed on a single line.

   "Minute" is a number from 0 to 59. "Hour" is a number from 0 to 23. They represent the time of the day in a 24-hour day format, so for example, if you want a certain command to run at 5.30 am, you will have to code it as:

   30 5

   If you want something run at 8 pm, it has to be coded as

   0 20

   since 2000 hours is 8 pm in the 24-hour time format.

   "Day" and "month" refer to dates. "Day" takes a value between 1 and 31, and "month", as you may have already guessed, can take any value between 1 and 12. So if you want a command run on 5th January at 9.15 am, your schedule should begin with the following:

   15 9 5 1

   "Day-of-week" means basically which day you want your command to run. If you want your command to run on Sundays, use either 0 or 7 here. If you want it on Monday, use 1. (Note: if you are getting worried at this point how to combine all the various fields, some of which seem to contradict the other, don't worry. We're getting to that.)

   The trick to scheduling things, say, once a day, or once in 2 hours or the like, is to use a wildcard character. A wildcard character is like the Joker in a pack of playing cards, that is, it is something that can represent any card in the pack. In a crontab file, the wildcard character "*" (the asterisk, without the quotes), represents every possible value for the field.

   If you want a particular program to run, say, once every day at 10.45 am, the time portion of the cron schedule should read:

   45 10 * * *

   Here's how to read the above line.

   The first two fields "45 10" means that you want it to run at 10.45. The next field, the day field, is set to * (the asterisk character) to show that we're talking about 10.45 every day, not just the 1st of the month (which would be "1") or the 30th of the month ("30") or some other number.

   The month field is set to the asterisk as well. If we set some number in the month field, say "2", we will be saying that we only want the command to run at 10.45 in the month of February ("2"). Since that's not what we need, we put the asterisk to mean every month.

   Similarly, the day-of-week field is set to the asterisk, because we want the command to run whether it's Sunday ("0") or Monday ("1") or whatever day.

   More Examples: An Hourly Schedule

   Now if you want a job to run every hour on the hour, you will have to set the time component of the crontab line as follows:

   0 * * * *

   Can you see why? The "0" means at the top of the hour, that is, when the minute readout on a digital clock shows "00". The asterisk in the hour field means every single hour. In other words, every hour, on the hour.

   Alternate Hour or 3 Hourly Schedule

   If you want something to run once every two hours, you will have to use the slash, "/", character in your field. The slash character is the "step" character. In the case of a two hourly schedule, your time component of your cron file will read:

   0 */2 * * *

   The second field, "*/2", means every alternate hour.

   Similarly, if you want something to run every 3 hours, you can change that field to "*/3", and so on.

   Other Examples

   If you want a particular command to run only at 8.00am on the 1st and 20th of every month, you should code the time as:

   0 8 1,20 * *

   The comma, ",", means "and". If you are confused by the above line, remember that spaces are the field separators, not commas.

   What does the following schedule mean?

   2 3 4,5 6 7

   Decoded, the above line says at 3:02 am on the 4th and 5th of June (6) and on every Sunday (7), run your program.

   There are other possibilities in the time fields, and I won't go through all of them, since you already know enough to be able to construct whatever schedule you need.

3. How to specify the command line

   The command-line-to-execute portion of the schedule is basically the command you want run at the specified time. For example, if you have a Perl script called "whatever.pl" that you want run every day at 11.30 am, your crontab schedule might read as follows:

   30 11 * * * /your/directory/whatever.pl

   If your script is one of those that must be called from a web browser, like "cron.php" on a Drupal installation, you will need to use a command called "wget". Technically speaking, wget is not really a browser, but it works adequately like one for our purpose here, which is to simply get the web server to run the script called "cron.php".

   30 11 * * * /usr/bin/wget http://www.example.com/cron.php

   In this case, you will need to specify the full path to wget since, for security reasons, the cron program (not the cron.php script) usually executes in a sort of clean environment, which means that it won't know how to find the command "wget" unless you tell it where it is.

   Another important thing to note about this crontab line that we're constructing is that the entire line, schedule and command to execute, must fit into one line. You cannot put it into two lines for aesthetic reasons even if your command is very long.

4. How to be notified of errors

   Since you are running your script as a scheduled task, there will be nobody there to view its output. By default, cron will send any output from the script in an email to you, if it knows your email address.

   If your script is very talkative, and issues all sort of information when it executes, you'll probably want to shut it up (unless you are starved for email messages). To do this, we need to send all the normal output to a place called "/dev/null" which is basically like a black hole. It accepts anything you dump there, but you will never see it again. In the case of our first example, modify the command line to read:

   30 11 * * * /your/directory/whatever.pl >/dev/null

   The ">" sign means to redirect every normal message sent to screen to whatever is next in the command line, which, in our case, is /dev/null. If your script is designed to work correctly in a Unix environment, only the normal output will be swallowed up. Error messages will still be processed by the cron program. This is desirable, since you will want to informed when something is wrong so that you can fix the problem.

   To receive the remaining unredirected messages, you will need to add another line to your crontab schedule to specify your email address. Use the following format:

   MAILTO=email@example.com
   30 11 * * * /your/directory/whatever.pl >/dev/null

   The MAILTO line must be on a separate line. It is optional. That is, you don't have to specify it if you don't want to. Depending on how your web host has set up the system, cron might still be able to successfully send you error messages. If you really don't want to hear from cron at all, you will need to make your MAILTO line look like this:

   MAILTO=""

   That is, after the equal sign, put two double quotation marks without any space between them.

5. Create a text file with all the necessary lines

   There are many ways to feed all the lines you've constructed so far to crontab. I think the least problematic way for newcomers is to create an ASCII text file on your own computer containing all the needed lines. Use an ASCII text editor to do this. If you use Windows, use the program called Notepad, found in the Accessories folder of your Start menu. Do not use Microsoft Office or Word or some such fancy word processor program.

   Open a new document with your text editor, and type in the MAILTO line and your schedule (or just your schedule alone if you don't want to be notified when an error occurs). Remember, everything must fit into one line. Do not allow your editor to wrap the line. After you have typed in your line, hit the ENTER (or RETURN key on the Mac) so that your cursor is on a new blank line just after your schedule. Read that last sentence again; it's important.

   Save the file using any name, but with a ".txt" extension. You especially need to do this if you are using a system like Windows which use a different line ending from Unix systems. The ".txt" extension will cause your FTP program to translate the line ending to a Unix line ending when it uploads the file. If you're not sure, just name the file "crontab.txt" (whatever system you happen to be using), and you should be fine.

6. Upload the file to your web server using an FTP program

   Now upload the file to your site using your FTP program. The file must be uploaded in ASCII mode, which your FTP program will probably automatically do when it sees that you're uploading a file with a ".txt" extension.

7. Connect to your web server's shell account

   Connect to your web host's shell using your telnet or SSH software. If you don't have one, download one from the Free SSH (Secure Shell) and Telnet Clients page. I will not go into the details of how you should use the software to log into your shell account, since the method varies according to the program you have downloaded. Ask your web host if you have problems.

8. Invoke crontab to set your cron job

   From the shell command line, go to where you uploaded your crontab.txt file. You can use "cd" (which means "change directory") to change to the appropriate directory. Then type in the line below followed by the ENTER key (or RETURN key on the Mac).

   crontab crontab.txt

   where crontab.txt is the name of the file you created and uploaded earlier.

   That command will cause the program "crontab" to install your cronjob. At the appropriate time, a separate program called "cron" will execute your job. If you are returned back to the shell prompt with no messages, it means all was well, and you can check that your schedule was correctly set by using the following command:

   crontab -l

   where the "-l" means list all the schedules or cronjobs that you have set so far. If you find you made a mistake, or if you wish to reschedule your tasks, you can remove the existing schedules with the following command:

   crontab -r

   You can add more schedules the same way. Put all your different schedules into a single crontab file. The first line should be your MAILTO line, followed by each schedule/command on a separate line. When you run "crontab crontab.txt" later, crontab will replace your existing schedule with the schedules in your new crontab.txt.

   Once you've completed everything, you can delete your crontab.txt file. It's no longer needed.

Dealing with Error Messages from Crontab

If you received an error message when you used "crontab crontab.txt", you probably made a mistake creating your crontab file. Here are the common errors:

1. You split your schedule/command line into more than one line. Every scheduled task must fit into a single line. Do not allow your text editor to wrap it to another line. Switch off the word wrap feature in your editor if it has one.

2. You did not end your file in a new line. That is, you failed to hit the ENTER (or RETURN) key after your last command line in the crontab.txt file.

3. You did not upload the file in ASCII mode. Name your file with a .txt extension to have your FTP program automatically convert it. Alternatively, learn how to change the FTP upload mode to ASCII.

4. You used a word processor to create your crontab file. If the editor you were using allows you to do things like underline text, make nice bullet points, or do anything fancy besides typing raw text, you're probably using the wrong program.

Conclusion

If all goes well, congratulations. You have mastered the syntax of this most arcane of schedulers and set a schedule for your tasks.