Web Conference 2004 Writing Perl/CGI Scripts for ITS/ASET Web services Advanced Debugging Techniques

`Backticking` buggy scripts

Among the serveral ways to call other programs such as system("program"), open(PROG,"|program_accepting_input") and fork()/exec("program"), a convenient way to call other programs is with backticks `like_this.cgi`;. The output from backticks is returned, and can be passed into a variable, or fed into a print statement.

If a CGI program doesn't complete the HTTP headers, die()s or writes other error messages to STDERR, it can be difficult troubleshoot without the error logs. There is a quick and easy solution to test buggy CGI scripts from perl. For example, if your passingdata.cgi script was failing with "500 - Internal Server Error", you could write another script like the following:

#!/usr/local/bin/perl

print "Content-Type: text/plain\n\n";
print "Testing passingdata.cgi:\n";

print `./passingdata.cgi 2>&1`;

This new script will write the proper HTTP headers and then call passingdata.cgi on the command line. Any GET parameters that are placed in the QUERY_STRING environment variable will be inherited by the called script. Other environment variables such as Cookies and REMOTE_USER will also be passed. The called script will also inherit the STDIN of the calling script, which passes POST parameters. If passingdata.cgi exits immediately, fails to run at all, or writes any errors what-so-ever, you will see it. You will notice the SQLite example script uses this technique to call sql_internal.perl.

The 2>&1 part instructs the STDERR messages to be merged together with STDOUT. The ./ indicates that passingdata.cgi is in the local folder.

If you set the Content-Type header to text/plain, you will see the HTML code unrendered. If you prefer to see the HTML rendered by your browser, switch it back to text/html. Some errors are easier to see with text/plain.

Alternatively, if your script runs, but you just want to include the STDERR in the output of your script, you can use:

#!/usr/local/bin/perl

print `./passingdata.cgi 2>&1`;

Telnetting to a Web server

The HTTP protocol is based on text codes. As you saw how HTTP is passed between a browser and a Web server on speakingcgi.html, it is possible to type an HTTP request line on the keyboard without any special characters. HTTP uses the Transmission Control Protocol (TCP) just like telnet. Telnet is a protocol like HTTP, and a set of client/server programs that use it. You can use a telnet program to connect directly to a Web server and speak HTTP with it. When was the last time you talked shop with a server? Now is your chance. Doing so reveals a lot of the inner workings of the HTTP protocol and can be very helpful in identifying bugs in CGI programs.

Windows ships with a telnet program.

1. Run cmd.exe from "Start" -> "Programs" -> "Accessories" -> "Command Prompt".
2. Type "telnet test.scripts.psu.edu 80"
3. At the blank screen, type "GET /users/x/y/xyz123/scripts/passingdata.cgi?color=yellow HTTP/1.0" and hit return twice
4. The output from your script along with the full HTTP headers are returned.

Hyperterminal can also telnet to servers

1. Run Hyerterminal from "Start" -> "Programs" -> "Accessories" -> "Communications"
2. If asked, supply any area code you wish, this is not needed for telnet.
3. Create a new profile, call it PSU or whatever you want.
4. In the "Connect To" dialogue, set "Conenct using:" to "TCP/IP (Winsock)"
5. Type in the "Host address:" test.scripts.psu.edu
6. Set the "Port number:" to 80, press OK
7. Disconnect - you will need to set additional settings
8. Under "File" -> "Properties", click the "Settings" tab
9. Click the "ASCII Setup" button
10. Check the boxes for "Send line ends with line feeds" and "Echo typed characters locally"
11. Save the changes, and hit enter to connect
12. Repeat the HTTP request line from the previous set of instructions

Other HTTP commands to try

• HEAD / HTTP/1.0 - HEAD is a method that asks for the headers only

• POST data - Here is how to send post data:

  POST /users/x/y/xyz123/scripts/passingdata.cgi HTTP/1.0 Content-Length: 9 color=red

  Notice the Content-Length header. This needs to be set to the exact number of bytes you are typing in the HTTP body - the POST parameters line.

"Telnetting" to an SSL secured Web server

Secure Socket Layer (SSL) encryption "wraps" the HTTP protocol, protecting it from network eavesdroppers. However this makes it next to impossible to read over telnet (which is partially the point). However there is a telnet-like program that can help you talk to encrypted servers.

The OpenSSL (http://www.openssl.org/) organization has built an encryption toolkit that is free for downloading, compiling and using for various encryption tasks, such as signing certificates and securing Web servers (we use it to secure our servers). It includes a command line tool that can do many tasks, including connecting to a SSL protected server (HTTPS). There is even a pre-compiled Windows version with an easy to use installer.

Once the toolkit is installed, you can connect to a secured server with the following command:

C:\OpenSSL\> openssl s_client -connect www.work.psu.edu:443

You will see a lot of SSL related messages fly by, displaying the settings in the certificate. Once they have subsided, you may type your HTTP request as with telnet.

If you have any questions, feel free to ask me - mailto:jcd@psu.edu

Content by: Jeff D'Angelo <jcd@psu.edu> © 2004

Last update on: Sunday, 13-Jun-2004 23:36:55 EDT