The Linux Guide Online

Chapter 04 - The FTP and HTTP servers

4.1 FTP

Using the FTP or the File Transfer Protocol is a popular way to transfer files from machine to machine across a network. There are a number of clients and servers using this protocol, thus making FTP and very convenient and universal way if file transfer.

There are two purposes of configuring a FTP server. One as a private server only, for the purpose of FTP access for the users on the machine itself. The other is the anonymous FTP that is configured to allow remote machines to access the FTP server and thus conduct file transfers.

4.1.1 Getting and Installing the FTP Server

Red Hat uses the freely available wu-ftpd server. It comes as an RPM package in all installations. Run the following as root, substituting the correct name of the RPM that you have access to. Make sure that you have mounted the CD-ROM containing the RPMS first.

rpm -ivh wu-ftpd-2.4.2b17-2.i386.rpm

If you plan to allow an anonymously accessible site make sure you install the anonftp rpm as well. The rpm will be know as a version variant of anonftp-2.5-1.i386.rpm. Now you have a working anonymous FTP server. Of course you need an active Internet connection, and a valid host and domain name for your machine to be truly accessible to the Internet. To truly complete the complete installation of the FTP you also need to install the FTP client, which comes as part of the ftp rpm package. Find and install the package too so that you have the complete suite of the server, the daemon and the client.

To test if the connection is working, ftp you own machine. (you may use your machine name or use the loopback IP address). At the login prompt login as anonymous and give your e-mail as the password. As you might have guessed this is a standard way of logging into a remote machine where you have no login. (logging in as user ftp is another such method) But note that this feature can be disabled by the administrator, in which case you need to have a valid login to be able to access the machine. Type bye at the prompt to quit the client.

4.1.2 How the FTP server works

Some of the knowledgeable users must have noticed that we did not start the daemon explicitly after the installation. This is because the ftpd does not by itself respond to requests. There is a central entity called the inetd (daemon) that acts as a pipeline, listening to requests and then forwarding them to the corresponding handler. (It also listens for request like the http, pop etc). The inet daemon therefore invokes the ftpd when someone connects to the FTP port. (port 21 is generally associated with FTP). The corresponding configuration file /etc/inetd.conf has the required line that allows this to happen.

After the server is initiated (by inetd) the client needs to provide a username and a password. This is for the basic authentication. As already specified earlier two special login names - anonymous and ftp also exist and are treated differently. If a user has an account, and additional check is performed on the system, to ensure that the user has a valid shell. Users without a shell are denied access. This therefore acts as a special authentification that allows access through say the POP but deny access through FTP or TELNET. Normal users are put into their home directories upon login and the special users land up in the home for the FTP users, normally /home/ftp/.

Then a call to chroot is performed by the system that in effect blocks the user from going beyond the /home/ftp effectively sealing most of the sensitive directories from the users. To make up, there is a there are a set of /bin and /lib files for the user created under the /home/ftp are placed so that the system can access them even after a call to chroot. Most FTP sites have a /pub subdirectory under the ftp root that is used by the administrator to place files that can be accessed by the users.

4.1.3 Configuring the FTP server

Although the configuration the standard server is reasonably secure, you can fine tune access rights by editing the following files

  • /etc/ftpaccess
  • /etc/ftpconversions
  • /etc/ftphosts
  • /etc/log/xferlog

These allow you to control the who, and where of any FTP connection and also allow you to have an audit trail of what they do after they connect.

Controlling access /etc/ftpaccess file

The /etc/ftpaccess is the primary means of controlling who and how many users can access your server. Each line in the file either defines an attribute or sets its value.

The following commands control access

class

The class command defines a class of users that can be accessed later with the help of a single keyword. The class of users may later have specific permission set or removed from them. The form is

class <classname> <typelist> <addrglob> [<addrglob>]

The classname is the keyword to identify the class. The typelist is the type of clients, which may be any of anonymous, ftp, real. The addrglob is the Range of IP addresses that fall in the class. The range can be defined using the '*' sign.

class anonclass anonymous *

refers to the entire anonymous users, while

class ourclass anonymous 202.141.76.*

refers to the IP addresses that have the same first three numbers as defined.

autogroup

This command is used to control the anonymous users in sets more tightly by automatically assigning them a certain group permissions when they log in. The format is

autogroup <groupname> <class> [<class>]

where groupname is a group defined in the /etc/groups file and the class is the as defined using the class directive. Note again that these affect will be restricted only to the anonymous users referenced by the class.

deny

The deny command explicitly denies access to users from a particular domain or IP address range and also displays a message for the denial. The format for the command is

deny <addrglob> <messagefile>

An example to deny the users of hack.com from accessing your site will be

deny hack.com /home/ftp/messages/nohack.txt

guestgroup

This command is very useful when you have real users but want them to have only restrictive FTP privileges. The format for the command is

guestgroup <groupname> [<groupname>]

where groupname is the name of a group as defined in the /etc/groups. The effect of the restriction is that the user is treated more like an anonymous visitor.

limit

This command enables you to control the number of users that are connected to your system at any time. The restrictions can also be placed for class of users and also as various times of the day. The format is

limit <class> <n> <times> <messagefile>

where class is the class as defined using the class directive, n is the maximum limit, times is the time where the limit is to be enforced and messagefile is the file to be displayed in case the maximum limit is exceeded. All the arguments except the times are very self explanatory. The times is a comma delimited string where each option is for a separate day. The days are designated as Su, Mo, Tu, We, Th, Fr and Sa, and Wk referring to the whole week. Time is in the 24 hr clock without a colon separation.

For example to limit all anonymous users to 10 from Monday through Thursday, and till 5pm on Fridays (basically the working hours) you will use the command

limit anonclass 10 MoTuWeTh,Fr0000-1700 /home/ftp/messages/toomay.txt

loginfails

This command allows you to set the number of failed login attempts before the clients are disconnected. This is by default 5. This command is very useful when someone is making use of a hacking program which is trying to guess passwords by brute force. The format is

loginfails <n>

private

You might find it convenient to be able to share files with other users via FTP without having to place the file in a 100% public place or having to give these users a real account on the server. The clients use SITE GROUP and SITE PASS commands so that they can change to privileged groups that require passwords.
For your FTP to support this you need to set the private flag, using the command

private <switch>

Where the switch is either the string YES or NO to turn the support on and off respectively. Because you require passwords for these special groups, you need to use the /etc/groups file. The format for an access group is

access_group_name:encrypted_password:real_group

The following commands control the information that the server tells the clients

It is often useful to provide messages to FTP users when they connect to your site or specify an action. This provides not only for uniqueness of the site but also is a great way of documentation.

banner

This command specifies the message that must be displayed on the screen before the client has to provide the login and the password. This is an important opportunity to display your server policies and information to the anonymous users etc. The format of the command is

banner <path>

Where path is the complete path to the file whose contents have to be displayed.

email

The email is used to specify the site maintainer's email address. It is a good practice to use an FTP name and then alias it to the actual address.

email <address>

message

This command enables you to setup special messages to be sent to clients when they either log in or change to a certain directory. You can specify multiple messages. The format is

message <path> <when> {<class>}

where path is the name of the file to be displayed, when is the condition and class is the list of classes that this message is to be displayed to. The <when> parameter has one of the two forms: either LOGIN or CWD=<dir>. When the parameter is login, the message is displayed upon successful login. The second type displays the message when the corresponding directory becomes the current directory for the client. The class is optional.

The message file itself can contain special flags that are changed with the appropriate information at the time of display. These options start with a '%' sign and characters like T (local time), F (free space) etc. the total list is given below.

Option Description
%T` Local time
%F Free Space in the partition where <dir> is located
%C Current working directory
%E Site maintainers email (as specified by the email directive)
%R Client hostname
%L Server hostname
%U Username provided are login
%M Maximum number of users allowed of the particular class
%N Current number of users in the specified class

A sample command would be

message ./.toomany_anon LOGIN anonclass

where the file path is relative to the ftp home in case of anonymous users.

readme

The readme command enables you to specify the conditions under which clients are notified that a certain file in their current directory was last modified. This command has the form

readme <path> <when> <class>

The path is the file that you want the users to be notified about. When is similar to the option in the message command. And the class directives is the class for which this is applicable. The when and class commands are optional.

The following commands control the logging

log commands

Often for security purposes, you might want to log the actions of your FTP users, which you can do using this command. Each command invoked by the client is sent to your log file. The format is

log commands <typelist>

where typelist is a comma separated list specifying which kinds of users should be logged. The three types of users are recognized anonymous, ftp, and real.

log transfers

If you want only to log the transfers done by your users instead of the commands, then you can use this command. The format is

log transfers <typelist> <directions>

Where the directions refers to the direction the transfer must take to be logged, the two types being inbound and the outbound, referring to the uploading and downloading respectively.

The following are miscellaneous commands

alias

The aloas command enables you to define directory aliases for your FTP clients. These aliases are activated when the clients use the cd command and specify an alias. This cabability is seful to provide shortcuts to oftern requested files. The format is

alias <string> <dir>

cdpath

Similar the UNIX path environment variable. The cdpath command enables you to establish a list of paths to check whenever clients invoke the cd command. The format is

cdpath <dir>

Where the dir is the path of the server to be checked, relative to the FTP home. For example if you add the following paths

cdpath /pub/music
cdpath /pub/coffee

If the client enters the cd instant command, the server examines the following directories in the shown order.

1. ./instant
2. Alias called "instant"
3. /pub/music/instant
4. /pub/coffee/instant

compress

The wu-ftpd server offers a special compress feature that enables the server to compress or decompress a file before transition. With this capability a client who might not have the necessary software to decompress a file will get the file in the normal form. The form is

compress <switch> <classglob>

Where the switch is either YES or NO and the classglob is the class or classes to which the command applies. The additional configuration to be done is the change the /etc/ftpconversions file to specify what programs to use to the specified file extensions.

tar

Similar to the compress command for tarring or untarring user files. It has the format

tar <switch> <classglob>

chmod, delete, overwrite, rename, umask

These commands are used to specify if the class of users are allowed to perform the command on the files on the server. They have a similar syntax

<command> <switch> <typelist>

where command is the name of the command, and switch is either YES or NO. Typelist is a comma separated list of users who can execute this command. Most of the commands are self explanatory. The umask determines if the client can change their default permissions in a fashion similar to the shell command by the same name.

passwd-check

Providing a valid email address as a password is considered good manners when connecting to an anonymous FTP site. This command specifies how strict you want to be with the verification of the string that is submitted as a password. The format is

passwd-check <strictness> <enforcement>

Where the strictness is one of the three possible strings "none", "trivial" or "rfc822" and enforcement is one of the two strings "warn" or "enforce". All the options are self explanatory with the additional information that trivial only checks for the "@" symbol and the rfc822 requires the email to be correct in all respects.

path-filter

If you allow users to upload files via the FTP server, you might want to dictate what are the acceptable filenames. This is done by the path-filter command, with the format

path-filter <typelist> <mesg> <allowed-regexp> <denied-regexp>

where the typelist is one of "anonymous", "ftp" or "real"; mesg is the name of the file to be displayed if the file cannot be accepted.; allowed-regexp is the regular expression of the file names that are accepted and denied-regexp is that for those explicitly denied.

path-filter anonymous,guest /messages/badfile UL* sex*

upload

You can use the this command to specify what permission the client has to place files in certain directories, as well as what permissions the files will take on after they are placed there. The format is

upload <directory> <dirglob> <switch> <owner> <group> <mode> <mkdir>

where directory is that which the command applies to; dirglob is used to determine whether a subdirectory under the directory is a valid place to make an upload. Switch is either YES or NO. The next three options determine the permissions the file takes after being placed there. The last is to specify if the client can make directories, and is one of the two dirs or nodirs.

FTP administration commands

The following commands can be used to control the FTP server. The ftpshut command can be used to shutdown the FTP server when it is being run as a service all by itself without using the inetd. The ftpwho displays all the active users using ftp at the time. The ftpcount gives statistics on the number of users of each class. Read the man pages for more information on all these commands.

4.2 HTTP

4.2.1 Apache Server - Installation

The apache server which started as "a patchy server" has grown to be one of the most used web servers in the Internet today. The reason for its popularity has been it power, ease of use, security and it transparency of configuration. You can either find the latest Apache RPM either in your installation, or on the web. Obtain the latest version that you can lay your hands on because that will be the safest in terms of the number of bugs fixed, and security problems patched. Installing Apache from the source code is beyond this book. Obtain and install a RPM of the file. For installation use

rpm -ivh latest_apache.rpm

And for upgradation use

rpm -Uvh latest_apache.rpm

The apache installs in the following directories:

/etc/httpd/conf - the configuration files
/etc/rc.d/ - the tree that contains all the startup scripts that must be used to start Apache.
/home/httpd - this is the place where the cgi-bin and the icons directory is created. Also the /html directory which is the apache root in also created here with an apache users' manual.
/usr/doc and the /usr/man - contains the manual and the readme files.
/usr/sbin - the executable binaries.

4.2.2 Runtime Sever Configuration Settings

Before we go on and describe the apache server, it will be well worth saying that running an apache server without any special configuration at all is very possible. After the rpm is installed, restart your computer (or just run "/etc/rc.d/rc3.d/httpd start" command) for the server to start. Assuming that your computer is configured for a network (the hostnames, IP addresses and other configuration has been done), startup your favorite browser and enter your hostname (or the IP address) and watch the Apache welcome page load into the server. Presto your server is ready. If you are on the internet with a permanent IP address you may well start serving web pages.

Also there are a number of tools available to configure Apache without having to edit the configuration files. The Comanche is one such tool for the purpose of configuring the server with a GUI.

Apache reads its server configuration settings from the files access.conf, httpd.conf and srm.conf. All configurations start with a directive command along with some options. Sometimes the directives may be for particular directories whence they are enclosed within sections as shown

<Directory somedir/in/your/path>
directive option option
directive option option
</directive>

httpd.conf

This file contains directives that control how the sever runs, where its log files are kept, the user ID (UID) it runs under, the port it listens to etc. You may use most of the defaults and change only those that are actually applicable to your machine. Some of the important directives are as below:

ServerAdmin : This is the address of the server administrator.

User and Group : This is the UID and the group ID (GID) that the server runs under. You may use the defaults for it. Red Hat Linux automatically creates the required UID and GID during installation. But be careful while changing the ID. If you give the server special privileges they may be used by hackers to take control of your system or do something worse.

ServerName : The fully qualified name of the server. If this is not set the server will automatically configure itself to the canonical name.

ServerRoot : Normally /home/httpd/ this can be changed the any thing that you want keeping in mind that the permissions for the target directory allow the files to be read by the server. This is the absolute path to the server directory.

srm.conf

This file contains configurations to the locations of the Web document tree, CGI program directories and other resource configurations. You may use most of the default configurations that come with the installation.

DocumentRoot : Set this directive to the absolute path of your html document tree. This is the path from which the server will obtain the files. The default is /home/httpd/html.

UserDir : This directive defines the directory relative to a local user's home directory. It is relative because the user home directive itself will be dependant on the user. The default setting is public_html. Hence of user123 is a valid user in the machine acting as the server, and he has a file called index.html in the directory $HOME/public_html/. This file is accessible through the server (assuming the permissions are given to the directories preceding it etc.) as http://www.domain.com/~user123/index.html. This can also be used to access any documents available in a tree under this directory.

Access.conf

This file is the global access control file; it configures the type of access users have to your site and also the documents you make available. The default provides unrestricted access to all files in the DocumentRoot directory.

What you may want to do is probably prevent all kinds of Indexes and FollowSymlinks options for all directories. These are potential security problems that can allows users to 'escape' from the DocumentRoot restriction, and allow them to see the file listing in the directory.

Options that you set in the access.conf file can be overridden with the .htaccess file where you set the directives on a per-directory basis. You can disable all .htaccess overrides by setting the AllowOverride directive to None.

4.2.3 Starting and Stopping the Server

Red Hat Linux uses the System V style scripts for the purpose of starting and stopping services (also called daemons). The script for the httpd server daemon is the /etc/rc.d/init.d/httpd file. Execute this command from the command line with one of the following commandline options to change the server state or get information - start, stop, reload, restart, status. The reload only causes the configuration files to be re-read after modifications. The status prints the pids that are used by the server when it is executing.

The output of the status command (or the command ps -aux | grep httpd) shows that there are infact a number of threads of the server running. The httpd spawns into a number of processes before it starts to listen for connections. The number of such processes can also be configured.

4.2.4 Virtual Hosts

One of the most popular services to provide with a web server is to host a virtual domain (aka virtual host). A virtual domain is a complete Web site with its own domain name, hosted on the same machine as other sites. Apache implements this ability in a simple way, with directives in the httpd.conf file. Before configuring the Web server, you must configure the Red Hat system to handle multiple hosts.

Adding Virtual Hosts

Currently a big portion of the browsers surfing the net only understand http version 1.0. Why is this important? These browsers cannot distinguish virtual hosts served by a server running on a machine assigned only one IP address. For most of the world to see your virtual domain, each domain must have a unique IP number, and this IP number must be bound to the machine hosting the domain. For example, your Red Hat box can have the IP numbers 10.1.1.5, 10.1.1.6, 10.1.1.8 and 10.1.1.134 pointing to it. These IP addresses can be bound to www.virthost1.com (10.1.1.5), www.virthost2.com (10.1.1.6), and so on.

The first step in binding these IPs is to have a DNS entry on your name server pointing to them. A name record might have the following form:

Virtdomain1.com. IN SOA ns.netwharf.com. root.netwharf.com. (
1998020702
10800
3600
604800
86400)
;; Name servers
virtdomain1.org. IN NS ns.netwharf.com.
virtdomain1.org. IN NS rtp2.intrex.net.

localhost IN A 127.0.0.1
www IN A 10.1.1.5
www IN MX 10 ns.netwharf.com.
www IN MX 20 www.netwharf.com.
www IN MX 30 rtp2.intrex.net.
virtdomain1.com. IN MX 10 ns.netwharf.com.
virtdomain1.com. IN MX 20 www.netwharf.com.
virtdomain1.com. IN MX 30 rtp2.intrex.net.

This DNS record maps the IP 10.1.15 to the site www.virtdomain.com, so if you ping www.virtdomain.com. You see the IP 10.1.1.5.

The next step is to use the ifconfig command to bind the IP to the host onto a network device. This might not be a physical device as you will see in a few moments. You may use the following syntax:

/sbin/ifconfig eth0:count virtdomain

eth0 is the network device and count is the numeric value used to identify the host to the network system. virtdomain is the IP of the virtual host that you want to host on your computer. Then add a route to the new host so that other computers can find it. Use the route command. The following places the first virtual domain in place and adds the route:

/sbin/ifconfig eth0:1 10.1.1.5
/sbin/route add -host 10.1.1.5 dev eth0:1

That done, your system is capable of responding to different IP addresses. Thus the first step of configuration is done.

Configuring Virtual hosts

Once you have setup the physical computer to handlr virtual hosts, getting Apache to server them is simpler. Virtual hosts are configured used the VirtualHost directive in httpd.conf. They have the following format:

<VirtualHost www.virthost1.com>
DocumentRoot /home/httpdv1/html
TransferLog /home/httpdv1/logs/access.virthost1
ErrorLog /home/httpdv1/logs/error.virthost1
</VirtualHost>

The VirtualHost directive is used to define the Virtual Host. All directives inside this directive apply only to the requests made to the particular host name. Hence you can see that the new documents are being served from a directory slightly different to the standard Apache document root.

That is it. Restart httpd daemon and access the new site. (After of course putting the files, and permissions)

Conclusion

At this point you might have a properly configured Apache server running. Several advanced options are available, for which you might have to check out the online and other documentation. In fact having a server is no joy until you can harness the full performance of using the server side modules and other similar tools to enhance and make your server more performance oriented.

The first thing you should look at is the CGI programming through a language like the Perl. You may also want to look as developing SSI pages, or pages in PHP. The modules for most of these are available as rpm files that can be downloaded off the net. Finally note that Apache is not just restricted to Linux. Ports are available for OS/2 and even 32-bit Windows (98 & NT based).