License
Download
Documentation
Mailing list
News
Architecture
Contact: Alex Jacobson <AlexJacobson AT! HAppS.org>
HAppS is a framework for developing Internet services quickly,
deploying them easily, scaling them massively, and managing them
ziplessly. Web, persistence, mail, DNS and database servers are all
built-in so you can focus on app development rather than integrating
and babysitting lots of different servers/services (the Haskell type
system keeps everything consistent).
- HTTP Application Serving
Performs better than Apache/PHP in our informal benchmarks (thanks to Data.ByteString), handles large (video) files and lazy (javascript) streaming, supports HTTP-Auth, and more. It's part of your app so you don't need to deal with separate configuration and management of an HTTP server. Note: If you really need Apache on port 80 for some reason, it's easy to configure it to proxy to your HAppS app running on another port.
- SMTP Receiving (no more .procmail complexity)
Stop worrying about whether a separate mail server is up and stop dealing with .procmail or other user level inbound mail configuration hackery. HAppS can operate as an inbound SMTP server, converting inbound envelopes into just another event for your application to process. And, if you need a separate mail server on port 25, it should be much easier to configure it to SMTP relay mail to your HAppS app handling SMTP on a different port (you still avoid extra .procmail complexity/annoyance).
- Apps as Simple State Transformers
HAppS keeps your application development very simple. You represent state with the Haskell data structure you find most natural for that purpose. Your app then is just a set of state transformer functions (in the MACID Monad) that take an event and state as input and that evaluate to a new state, a response, and a (possibly null) set of sideeffects.
- XML/XSLT to Separate Application Logic and Presentation
HAppS lets you focus on application logic and lets you defer presentation entirely to XSLT, JSON, Flapjax, etc. HAppS converts automatically from inbound protocol level event types e.g. url-encoded HTTP requests to inbound application level event types e.g. ChangePassword. Similarly, it converts automatically from outbound application events like PasswordChanged and outbound protcol events like HTTP responses or SMTP messages. It even knows to apply XSLT server side for XML outbound SMTP messages and browsers that don't support XSLT client side. Currently, you still have to write instances for FromMessage and ToElement, but we hope to make that automatic soon.
- ACID Persistence, Concurrency. At-least-Once side-effects.
With HAppS you need don't to spend time marshalling data into and out of external RDBMSs to get ACID semantics (concurrent-access) for your data. HAppS treats all events as atomic and puts them in a total order so you never need to worry about concurrency (isolation). HAppS achieves durability by state checkpointing and write-ahead logging events. End-users can never be confused by a server reboot because HAppS won't execute responses or side effects until their driving events have been logged. HAppS also tracks which side-effects have completed. If the server is rebooted before a side-effect completes, HAppS will retry on recovery. (This sophisticated side-effect functionality may be unique to HAppS)
- (Experimental) Table and Index in Haskell
Do relational operations (type) safely on in-memory Haskell Data.Set(s) rather than dealing with an external SQL relational database. Define custom indices for your Haskell datatypes (e.g. geographic/geometric types). Use in combination with MACID for a robust relational DBMS customized for your application.
- Coming Soon: No need for server architecture (thanks to Amazon)
We are almost done with changes to the back end of HAppS so that apps will be able to run unchanged on Amazon's S3 and EC2. The result will be massive scalability and superior reliability without you having to lift a finger or walk into a data center.
Documentation
Tutorial
Module API documentation generated by Haddock.
HAppS was inspired by Python Twisted, Prevayler, and SEDA. For details, see HAppS Architecture.
For trivial usage examples see hello world and simple mail and http example. For the source to Pass.net, a live application built on top of HAppS:
The server:
darcs get http://www.pass.net/s/repo
The client library for using pass.net authentication in your application:
darcs get http://pass.net/s/UsePassNetClient
Current issues
- MailSender may run out of sockets upon restart.
- Some problems with zombie threads (good bug reports appreciated).
License
HAppS is released by HAppS.org under the terms of
the BSD3 license which is enclosed with this package.
How do I get HAppS?
News
Recent changes
Darcs changelog.
0.8.8
- Complete revamp of the http interface
- Simplified default serializing
- Easier use of IO in results
- Easy use of request and response filters
- Expanded and updated tutorial
0.8.4
- Use GHC 6.6 by default
- DecodeStringM in MonadIO
- Experimental blocking io, SessionKeeperEx, Users etc
- Many new examples
- StartStateEx for more complex StartState components
- MACID cleanups
0.8.2
- Switch to BSD3 license.
- Support for new ByteString.
- Support GHC 6.6 (rc-versions)
- Add new examples: cookie, dns, http.
- Good support for cookies.
- Revise HAppS.DNS and make it easier to access from the outside.
- Much faster URI parsing.
- Many internal changes to HTTP and Saver imlplementations.
Documentation Update (April 13, 2006)
Modified top of the homepage to reflect current thoughts on the value of HAppS.
0.8 GIGANTIC REFACTOR!!! (April 10, 2006)
Einar Karttunen has basically completely rewritten the ACID and HTTP
codebase. Most of my code is gone. Einar's new code exposes a really easy
to use monadic interface for apps. I've added some useful features.
- New Monadic Interface
- Live App using code is available via:
darcs get http://happs.org/pass.net
- FastPackedString means really fast HTTP performance
- Server side xslt support (via xsltproc) for http user-agents that can't do it client side
- DNS resolver so no need for local DNS configuration
- Mail sending agent does exponential backoff for mail delivery so need for local mail server
- XSLTMail (using xsltproc) so mail can be formatted using XSLT rather than in code (like HTTP)
- Easy to use server session library
- StdMain automatically produces all the code you need to run everything
- Easy multi-server support (combined mail and http server without TwoListen hassle)
And much much more.
General Commentary
HAppS.MACID
Use HAppS.MACID to build applications that need
to survive random shutdowns.
- Atomicity:
You provide a state transformer function. Any errors mean
that state does not get updated.
- Consistency:
Via Haskell's type system.
- Isolation:
All events arrive on a queue and are processed in total order.
(Note: Haskell's concurrency model combines with its referential
transparency to make total order vastly superior to other processing models.)
- Durability:
Via write ahead logging of events and periodic serialization of
state. In the event of a shutdown, the log is played back from the
last save.
HAppS.DBMS
Use HAppS.DBMS if you want
relational database like functionality for your Haskell types without
using the IO Monad.
- HAppS.DBMS.Example
- This file provides a good explanation of how to use Haskell.DBMS
- HAppS.DBMS.Table
-
You can think of
a HAppS.DBMS.Table as acting like a Set with queryable indexes. A
database then is simply a datastructure with one or more of these
tables. The module provides you with useful functions for operating
on tables including union, intersection, difference,
innerJoin, outerJoin, agggregate, groupBy, etc.
A Table is actually a class rather than a datatype. Until someone
implements the TemplateHaskell (hint! hint!), you will need to
instantiate the class manually. See HAppS.DBMS.Example
for an example of setting up and querying your database.
- HAppS.DBMS.Index
- Contains code
for indexing various types of data. An Index is the reverse of a
Map. You look up element values and get back sets of keys that
have those values. Currently two basic indexes are implemented on
top of Data.Map. One handles Ord lookups for any element type that
supports Ord. The other provides really basic text indexing.
On HAppS Architecture
HAppS design was inspired by other work
It uses Concurrent
Haskell's select/poll style threading model to decrease resources
required to service each client.
It follows SEDA's lesson
that events should accumulate in their entirety in queues before they are
processed.
Prevayler
style write-ahead logging of queued events and state checkpointing means
it does not have to rely on an external ACID database for integrity.
(The big insight here is that, because protocols like HTTP and SMTP are atomic,
we don't need all the support for long transactions in the back end system)
Transaction
serialization means no concurrency/locking hairballs for
developers
Referential
Transparency allows checkpointing w/o requring app. lock (solves
Prevayler's big problem with serialization performance.)
Prospective Good News
Haskell is
more expressive than other languages so you can create DSLs for your
application domain and define business logic more efficiently.
Having the
DBMS as part of the app means you don't have to write all the
database marshalling/unmarshalling code typical of LAMP. So you can
develop much faster.
The type
system means that it is difficult or impossible to make a large
class of errors that are incredibly typical of traditional database
programming. So you are less likely to run into costly run time
errors.
Integrating
the pieces together in memory means you are not expending the huge
amounts of CPU time used in LAMPs marshalling data between the HTTP
server and the app server and then the app server and the DBMS and
the underlying disk. So you can save money on buying, maintaining,
and hosting large numbers of CPUs.
Integrating the pieces together also
means that you deploy a single integrated and tested executable and
are not as much at the mercy of the vagaries of the deployment
environment. So you can save money on sysadmins.
What applications benefit from HAppS?
HTTP
requests and SMTP envelopes encapsulate transactions and not vice
versa.
Note: doing otherwise with LAMP is considered bad
design because it implies a requirement to maintain and garbage collect
database connections arbitrarily. So this should not be a high
hurdle.
All
operating data fits in memory (store blobs on disk.)
Note:
Although this seems like a high hurdle, COTS servers with 12gb of
memory are readily accessible and some vendors let you reach up to
500gb of RAM. FYI, eBay has around 50M active users. If you
maintained 1k of queryable data for each of their users, you would
need only 50GB. (You would also need to recompile your app for
64bits so the math is a little more involved but you get my point).
You don't need more CPU power
to server your app than you can obtain in a single machine.
Note:
I have not benchmarked this code yet, but another Haskell server was
benchmarked at near 1000 HTTP transactions per second on a Pentium 4
in 2000. Modern web servers with similar architecture can serve 10k
HTTP transactions per second. eBay serves 400M page views per day,
which comes to an average load of 5000 hps and a peak load of
perhaps 50k hps. In other words, an OTS 8 CPUs system, could handle
all of eBay's HTTP traffic.
I am not saying that using HAppS, you
could serve all of eBay on a single box. I am saying that your
application is likely to be well within the constraints required for
HAppS to make sense for it.
Todo
User agent detection needs refinement. Currently defaults to client XSLT. Should be the other way.
Disclaimer
THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
WARRANTIES, INCLUDING, BUT NOT LIMITED TO, ANY IMPLIED WARRANTIES
OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
Copyright (c) 2006 HAppS.org. All Rights Reserved.
Contact: Alex Jacobson <AlexJacobson AT! HAppS.org>