Shid_httpd

Name:     SHID - Simple  HTTP Internet Daemon
Author:   DaveP from AmigaWorld.net
Status:   Alpha, at your own risk
License:  Freely copyable in this version
Version:  0.6.2.a.ppc.amigaos4
Today:    02/May/2005

Special note:

This release is intended to fix many minor bugs, give you a sample that
does most of the SHID work that you can adapt for the SDK as well as adding 
some missing header files.

It also might have a chance of working with the updated php-cgi binary
thats in the works...

It also includes a more convenient way of stopping and starting not only
the entire web server process and its plugins but individually too. See
notes below!

To allow apply/save style configuration changes it first attempts to read
settings from env and then envarc ( the second part is redundant but I
left the code in for now ). Thus, if you wish to try out a new configuration
without changing it write a configuration file in Env: for it and bounce
(stop and restart ) the http process involved.

Introduction

This is an introductory version of httpd for AmigaOS4.0 and is a total
ground up write of an http server. This introductory version is shown
to work reasonably well but has the problems listed in 'BUGS' section.

It provides support for user written plugin modules, configuration
files, multi-threading and interpreted code such as PHP.

It is provided with the following configurable modules:

   * httpd_modcgi    - a CGI interface 

   * httpd_modfile    - a file i/o interface

   * httpd_modvhost   - a virtual server gateway, not yet documented
                        but the conf file should be vhost.conf and
                        the virtual server to port assigns should be
                        written like so in the conf file:

                        www.myhost.com:Default.Gateway
                        www.phpserver.org:Default.PHP

                        etc. ( not tested )

   * httpd_modphp     - an adapted CGI interface that is designed to
                        work with the flawed version of php-cgi on
                        os4depot.net ( see bugs ).

Also, to check that your cgi installation is working properly copy the 
cgitest binary into the cgibin: assign and call it with:

http://127.0.0.1/cgi-bin/cgitest/blah/wibble/ftang/shrubbery

To get some idea of where the direction is for this free utility please
check out Todo.

Its nowhere near where I'd hoped, but Ive been doing other work and as
the complexity has grown the burden of testing has too.

Prerequisites
=============

Amiga OS4.0 with php4.3.0.tar.gz from os4depot installed. If you do not
install php then remove the php definition from file.conf ( see below )
and edit httpd_setup and envarc:httpd/httpd/startup.conf to remove PHP entries.

Installation
============

Copy the env-archive directory contents to envarc:
Edit the file envarc:
Edit the configuration files to suit your preferences:
  * envarc:httpd/httpd/startup.conf
  * envarc:httpd/httpd/httpd.conf 
  * envarc:httpd/Default.Gateway/file.conf
  * envarc:httpd/Default.PHP/php.conf
  * envarc:httpd/Default.CGI/cgi.conf

Run using 'execute' ( as its a shell script ) httpd_setup
Run directly ( as its a binary ) httpd_start to run all the entries in
startup.conf

To stop httpd, run httpd_stop.

To start or stop one or more plugins list their port names after the command,
e.g.:

httpd_start Default.Gateway Default.PHP
httpd_stop Default.CGI Default.Gateway

The default options will be sufficient to provide a web server that
serves files and PHP and copes with CGI binaries.

For more complex installations please refer to the user manual in the
docs directory of the installation file.

Bugs
====

* Non conforming - There are still some defects in conforming to the
  HTTP1.1 and HTTP1.0 spec.

* Some odd unexplainable lockups, yellow gurus and the like ( bah ).

* Escaping out of the filesystem - The filesystem plugin does not check
  you aren't parenting out of a assign and into its parent so either
  use a partition for each filesystem OR don't use it if you are
  even slightly concerned.

* Whilst you can allow or deny HEAD, GET, POST etc only GET has been tested
  and GET is currently not 'deniable'.

* Threading still hasn't been tested yet.

Todo
====

* Tighten up to conform properly with the HTTP/1.0 spec and HTTP/1.1 (
  important for keep-alive ).
* AmigaGuide manual.
* Optimise, optimise, optimise ( lots of scope here ).
* Apache style logfiles.
* POST/HEAD testing, fixing and enablement.
* Integrated php module.

Warning
=======

This is a very alpha state, that means its not really of the quality I
want it to be, its not fast, it has problems and can be brought down
with some lateral thinking. So please do not use it in a production
environment yet.

See the warning about 'escaping' in the Bugs list.

At the moment configuration is done through flat files, as is detailed
in 'Configuration' in the manual and complex configuration writers
should be prepared for the next version to use XML instead ( which was
put off this time because of lack of the sax parser required! ).

Do not whatever you do think there is any warranty of any kind with this
code. If you don't accept it, don't use it.

Notes
=====

There are several serialisation points, as each module in the server
uses IDCMP to a subtask if that subtask is not multi-threaded, then
under load, multiple requests can get backed up. So it is important
that the subtask ( module writer ) programmer either learns about
pthreads OR about subtask gateways to broker requests properly.

Features in this version
========================

[_] Multiple port handling
[*] Virtual servername handling
[*] Interchange protocol for gateway writers
[*] Basic file I/O gateway
[_] Fast-PHP4 gateway
[*] Slow PHP4 gateway
[*] CGI gateway
[_] Fast-Perl gateway
[*] Developer includes and library for gateway writers
[*] Server startup
[*] Server stop
[*] Plugin start/stop
[_] Multi-threading
[_] Multi-tasking
[*] Documentation
[_] Locale
[_] XML configuration files
[_] Installer script
[*] Mime handlers
[_] ErrorDoc handlers
[_] De-facto standard logfiles.


Features in next available public version
=========================================

[_] Multiple port handling
[*] Virtual servername handling 
[*] Interchange protocol for gateway writers
[*] Basic file I/O gateway
[*] Fast-PHP4 gateway
[*] CGI gateway
[*] Fast-Perl gateway
[*] Developer includes and library for gateway writers
[*] Server startup/stop/ plugin start/stop
[*] Multi-threading
[*] Multi-tasking
[*] Documentation
[*] Locale
[*] XML configuration files
[*] Installer script
[*] Mime handlers
[*] ErrorDoc handlers
[_] De-facto standard logfiles.

( hopefully )

Goal
====

To be the fastest you can get on AmigaOS4, to have the most features,
to be ground up scratch 'Amiga' rather than yet-another-UNIX port.

Design
======

-----> [ port ]--[daemon]
                 [ thread pool ]
<------------------[ thread ]
                     |    |
                    /|  |/
                     |    |
         [ IDCMP route to and from plugin ]
                     |    |
                    /|  |/
                     |    |
              [httpd_pluginutils.o]
                        /
                  [ plugins ]

            /     /    |            
         [cgi] [php] [file] [perl] [vhost]

Configuration Files
===================

Envarc:httpd/
	  httpd/
                  httpd.conf
	          startup.conf
          [gatewayname]/
                        vhost.conf
                        file.conf
                        cgi.conf
                        php.conf

Configuration Files
===================

Configuration files do not use XML at the moment ( because I do not
have the developer includes at time of publication for the parser
library I intend to use ). They are a plain text file of lines with
properties and values seperated by a ':' character.

The configuration properties and values are case sensitive, so be
warned and you cannot yet place a space between the property and
the value so whilst:

Property:Value

...works, the following doesnt:

Property: Value
Property :Value
Property : Value

You can use the : character in the Value section as only the first :
is checked for.

# is a comment character, a line that STARTS with that character gets
ignored and can be used for marking up information in the file.

Empty lines are ignored.

Configuration files other than httpd.conf are looked for in the port
name subdirectory under envarc:httpd/ followed by the specific
configuration file.


httpd_modfile Configuration
===========================

Envarc:httpd/[port]/file.conf

To run http_modfile you need to provide it a single argument that is
the port that it will listen on. If you do not provide a configuration
file for it in Envarc:httpd/[port]/file.conf then the default
configuration will be used:

No MIME handlers.
No Interpreter handlers.
Only GET allowed as an HTTP method.
Assign:HTTPDOCS:
CgiPath:/cgi-bin
CgiHandler:Default.CGI
DefaultPage:index.html

Mime:[extent]=[mimetype]
e.g. Mime:gif=image/gif

     Mime defines the type that is returned in the header information
     for a file based on its extent. Therefore if the browser requests
     a file wibble.gif then the Mime type that gets returned will be
     image/gif.
     Declare one for each Mime type you wish to support.

InterpreterExtentKey:[extent]=[gatewayport]
e.g. InterpreterExtentKey:php=PHP.Gateway

     InterpreterExtentKey calls another plugin that is defined to
     follow the interpreter plugin guidelines as provided in the
     developer documentation. In this case information is provided
     to the downstream interpreter which needs to be defined to
     listen on the gateway port you define.

Allow:[POST|HEAD|GET]
e.g. Allow:POST

     Which HTTP methods you wish to permit through. By default HEAD
     and POST are denied, but the code is missing to check this and
     enforce it at the moment.

Assign:[assign]
e.g. Assign:HTTPDOCS:

     The assign that will be used as the base of the filesystem that
     this module will serve up (and pass to downstream interpreter
     plugins ).

CgiPath:[path]
e.g. CgiPath:/cgi-path

     The path data that will be detected at the start of the request
     string to redirect it to a

CgiHandler:[Default.CGI]
     The name of the port where cgi requests should be redirected.

DefaultPage:[filename]
     The name of file to serve up by default if the request ends in
     a '/'

httpd_modvhost Configuration
============================

Envarc:httpd/[port]/vhost.conf

In HTTP/1.1 the Host: parameter indicates the host that was requested so
that multiple websites can be served from one host. This is NOT tested.

The configuration file contains multiple entries of all the host strings
you are expecting and how to interpret them.

[host]:[portname]

[ Note: Not tested ]

httpd_modphp Configuration
==========================

Envarc:httpd/[port]/php.conf

FileStem:[filestem]
     The prefix to use on temporary files generated by php execution.

TempPath:[assign]
     The assign where to store temporary files, keep it RAM:

PHPBinaryPath:[assign]
     Where the php cgi binary can be found on your disk.

PHPBinary:[executable]
     The name of the php binary to execute, found in PHPBinaryPath

httpd_modcgi Configuration
==========================

Envarc:httpd/[port]/cgi.conf

FileStem:[filestem]
     The prefix to use on temporary files generated by cgi execution.

TempPath: [assign]
    The assign where to store temporary files, keep it RAM

CGIBinaryPath: [assign]
    Where to look for cgi binaries.


httpd Configuration
===================

Envarc:httpd/httpd.conf

PortNumber:[numeric]
      The port number to listen on for incoming requests.

ThreadPoolMax:[numeric]
      The maximum amount of concurrent threads of execution allowed.
      ** IGNORED **

Gateway:[nameofport]
      The name of the message port to send requests to.

MaxListenerQueue:[numeric]
      The number of TCP/IP requests we will allow to pile up.

httpd_startup and httpd_stop global configuration
=================================

The format of this file is [port]:executable.

The important one to leave alone is 
SHID.CONTROL:httpd

After that its just : pairs for [port]:[plugin] which will
be used to launch plugins.

e.g.

Default.PHP:httpd_modphp


Developing plugins
==================

Whilst the plugin mechanism involves each plugin waiting on a message port
(and yes you can make it multi-threaded) accepting a request message from
httpd and serving back mini-response messages through an interface it is not
advisable that you follow this mechanism yourself without the SDK which will
be published seperately.

In the SDK wrappers for the I/O needed for POST and GET responses are
provided, dynamic hashmap API for user headers, upstream configuration and
other dynamic data as well as a configuration file handler which if written
to will guarantee compatibility with your plugin and any file format change
for plugins.

Process:
  - Create a message port and name it the way the argv[1] specifies.
  - Read the configuration file in envarc:httpd/<argv[1]>/[plugin.conf]
  - Wait for an incoming message.
  - Process that message.
  - Use the pluginutils.h functions to communicate with the browser.
  - Call CloseSocket when done. 
  - Return to  waiting for incoming messages.

When developing a plugin add the httpd include path in this archive
to your include path and link with libshid.a.

See the sample samples/sample_plugin.c which includes an easy framework
to handle most of the SHID mechanism leaving you to concentrate on the
fun bit.

Note on sizes
=============

Laziness has brought me down to the level of using __USE_INLINE__, as this
causes the executable to end up a bit larger than I want I'll be using
direct references and interface open/close explicitly from 1.0 onwards.

Note on bundling
================

If Hyperion want to use this in contributions drawer, they are more than
welcome although personally I'd wait for the php issues to be resolved. 

The next version IS 1.0.

If anyone else wants to bundle it, hard luck, you need to maintain the
archive in original format and it cannot be bundled as part of any other
software or sold etc. Be sensible, respect copyright and the author.