Internet Relay Chat (IRC) provides an international textual chat facility used by hundreds of thousands of people world wide. As IRC has grown, problems have become apparent both in the scalability of the implementation and in the network protocol it uses.
This paper describes these problems in more detail and also introduces new software (irc++) intended to address these problems.
The irc++ system is compatible with existing IRC clients, and also provides some MUSH/MUD/MOO-like facilities.
The main goal of irc++ is to provide a vastly scalable infrastructure for multimedia information exchange in real time.
Internet Relay Chat (IRC) was originally intended as a replacement for the Unix talk program, and allowed multi-way conversations between dozens of people at a time. It is now used by over a million people all around the world. This growth has not been without problems. The protocols do not scale well, and with the increase in usage come resource discovery problems and denial of service attacks.
Section One gives a brief overview of the current architecture of IRC and introduces some terminology and definitions. Section Two describes some of the main problems with IRC: that is, the ways in which IRC fails to satisfy its current users, and the ways in which it fails to meet a greater potential audience. Section Three introduces the irc++ architecture, and Section Four describes how the irc++ architecture is addressing the problems outlined in Section Two.
It is too soon to present firm conclusions; sample code exists for experimenting with user interface and backwards compatibility, but the main bulk or irc++ is, as of December 2000, unwritten. There has, however, been interest, and the irc++ web site http://www.holoweb.net/~liam/irc++/ will soon be hosting mailing lists as well as code downloads.
Like many open source projects, irc++ is also looking for a new name.
This section introduces the main concepts used in existing IRC implementations. Interested readers are referred to the IETF RFC 1459 that originally documented IRC, although that is not now adhered to precisely.
The primary purpose of IRC is to let people communicate by typing messages to each other in real time; IRC is often used for technical support as well as for chat, teenage dating, group design work, business meetings, even prayers.
In the current implementation, each user runs an IRC Client that connects to a remote IRC Server. A number of IRC Servers can be connected together to form an IRC Network.
Users join one or more Channels; they see the list of people in that channel, and can talk in the channel or send private messages to those people.
One user interface for IRC is shown in Figure 1.1. The program shown is the oddly-named BitchX IRC client, connected to a server on the SorceryNet network.
Users are identified by a nick; the author of this paper uses the nick Ankh for example. Nicks are limited to typically between 9 and 30 characters, depending on the IRC Network, and are not allowed to contain spaces, although digits and some punctuation are allowed. Nicks are global: only one person an an IRC network can be using a given nick at any one time.
Some networks run nick services, usually with a program whose interface is by sending messages to a privileged user with the nick NickServ. NickServ can associate a password with a nick, so that only users with the password are allowed to use the nick.
Channels are also given names, and these start with a # to distinguish them from nicks. Channel names are (in general) global, so that the list of channels is the same on all servers on any given IRC network, as is the list of nicks in each channel.
A channel operator is someone who can manipulate channel settings, including kicking unwanted people out of the channel, and possibly banning them from returning; channel operators can also change the channel topic, and make other people channel operators.
Some networks run channel services, which users can use to control who is allowed to enter a channel, and maintains a list of channel operators for each registered channel.
The IRC servers are connected together to form a minimum spanning tree; this architecture appears to have been chosen in the incorrect belief that it minimises network traffic over each link. An example IRC network is shown in Figure 1.2.
If a link is broken, the network is split into two disjoint trees; this is known as a netsplit. Users on one half of the network appear to have been disconnected from the point of view of users on the other half. Netsplits are very disruptive, and are one of the biggest problems with the current IRC infrastructure.
The routing of the network is controlled partly by configuration files (ircd.conf) and partly by IRC operators (opers) associated with one or more IRC servers. The IRC operators can reroute connections, although since this involves first disconnecting and then reconnecting a link, it causes a temporary netsplit.
This section describes some of the difficulties facing Internet Relay Chat today. These problems may be categorised as being cultural, that is, arising from opinions, beliefs and behaviours that are learned and shared among network users on IRC; architectural, arising from differences between the design of IRC and current usage, and technological, where there are limitations in the design. It is important to understand that all of these interact. People behave in ways that are influenced by the implementation of the software, and the implementation is of course affected by the architecture.
All IRC traffic is sent unencrypted over the Internet; this includes passwords for server connections. Networks that have user services generally have user passwords sent in clear text over the network.
Some networks use Netscape's Secure Socket Layer (SSL) between servers, but the connection between the IRC server and the user is unencrypted. A single-user system, such as a Linux workstation, could run a proxy server that encrypted the traffic, but most networks do not allow this.
SorceryNet supports MD5 authentication, in which passwords, once stored on the server, no longer need to be sent openly, but there is little or no IRC client support for this feature.
The Internet does not have a concept of the identity of a person; there is nothing to say that a user connected from home with a modem is the same person as someone connecting from an office, or even from the same computer using the same modem and ISP but with a different Internet Protocol address.
Users who wish to be anonymous because they want to say something controversial end up lumped in together with users who wish to be anonymous so that they can be obnoxious without being caught. the latter group is greatly the larger, but that does not invalidate the needs of the former group.
One example of abuse of anonymity is people who use gateways, shell accounts and bouncers so that their hostname is hidden; if they are banned from a channel, or even from the network, they then reconnect using a different hostname, and evade the ban.
Often, being obnoxious in this context means running a program that connects dozens or hundreds of distinct clients to an IRC server, and then has them join a channel and all speak garbage at the same time. This is easily detected; multiple clients with the same IP address are known as clones, and many networks automatically disconnect and ban them. Unfortunately, if the perpetrator uses a spoofed IP address, or a trojan program running a bouncer such as sub7 on someone else's computer, the ban will result in a legitimate user being banned.
Most modern IRC networks test each connecting user to see if they are running the most common form of bouncer, a SOCKS or WinGate proxy that can be used without a password, and, if so, disallow access to the network.
Anonymous users may also attempt attacks against other users. On many IRC networks, users' IP addresses are plainly visible, so that it is easy to attempt unauthorised access, or to send massive network packets to other users that will make their connection too slow for them to be able to chat.
If it seems that no responsible adult would do these things, it is useful to consider first that many children have network access, and secondly that IRC is often a mechanism of support and outreach for people who might have difficulty socialising in other ways.
As described above, Internet Relay Chat networks uses a minimum spanning tree topology. A leaf server can connect to at most one hub server, and there can be no loops or redundant connections. The result is that if any link is severed, the resulting netsplit creates two distinct sub-trees.
The network servers record the last time a message was seen from each connection (both client and server). If a certain amount of time passes with no message, they send a ping (not to be confused with ICMP query packets) down that connection, and, if there is no reply in a while, consider the connection to be dead and remove it. If the connection was to a server, this causes a netsplit.
Because the entire network state is replicated across all servers, a netsplit means that the servers on either side of the split then have to report that each user on any of the unreachable servers has quit; the resulting peak in bandwidth sometimes causes secondary netsplits. In addition, when the split heals and the server reconnects, the two servers that have newly connected must exchange information about all of the users on their respective sides of the network. Again, this connection burst can cause another netsplit.
IRC therefore becomes unstable when a network reaches a certain number of users with a given network bandwidth between the servers.
The oldest of the large IRC networks, EFnet, has a policy of non-intervention by staff: in other words, it is effectively unpoliced as far as most users are concerned. The culture of teenage cyberpunk is very visible. There are commonly available scripts for IRC clients such as mIRC and BitchX that will send well-known denial of service attacks to other IRC users; people who use these scripts without understanding how they work are often called skr1pt kiddies: they are often between eleven and fifteen years old, and like to use 31337 (elite) spelling, substituting numbers or symbols for letters in order to appear more knowledgeable, and failing.
Because there are no network services on EFnet, the only way to ensure that no-one uses one's favourite nickname is to remain connected all the time. One approach to this involves the Unix screen program, which lets a user detach from interactive programs and then later reconnect to the same program from a different login session. The result of this is that EFnet channels tend to have large numbers of clients in them, most of which do not correspond to users at their keyboards.
Several other networks (including DALnet) sprang up to cater for people who preferred an environment in which staff were helpful and friendly. Like all growing organisations, DALnet changed as it grew, and other networks sprang up to try to be more friendly. The technological problems are that IRC does not have a concept of an individual identity, and that IRC equates power and authority. Global IRC operators have essentially unrestricted power over the network in terms of their ability to disconnect or ban users,
IRC originally allowed users to exchange short messages in a shared forum; people quickly wanted to exchange longer texts, images, and, in time, sound and video. The IRC protocol is restricted to textual messages of up to 512 bytes, and cannot easily handle these things. People do, however, use IRC as a resource discovery mechanism: they find other users online to play games, exchange images and music files (and, sadly, warez), to give the address of shoutcast and icecast internet radio channels, and even to exchange video. But most of this is outside IRC. If a group of programmers want to discuss a UML class diagram, they can't use IRC in any standard way to annotate the diagram as they discuss it.
Furthermore, since IRC assumes a disjoint sequence of short messages, users cannot exchange documents or paste formatted paragraphs of text. The original scope of IRC was simply text; a separate protocol, DCC, later defined a way for one IRC client to communicate directly with another by exchanging IP addresses. It is thus possible to send files, and to chat on an out-of-band connection. The requirement that the IP address be shared is a security risk: a user can offer to send a file to a victim using DCC, and then use the IP address of the victim for a denial of service attack.
There are many problems with IRC, and yet there are over a million IRC users. The desire that people have to form social groups is greater than the annoyances caused by those problems. But that does not mean that things cannot be improved. There were millions of MS-DOS users too.
This section introduces the design of irc++. At the time of writing (December 2000), the irc++ architecture is not frozen; new people joining the project are contributing many valuable insights.
Irc++ introduces a number of concepts that are new to IRC. None of these concepts is entirely new to the Internet: this project is not intended as fundamental computer science research at the protocol level, but rather, as an experiment in social engineering, in providing support for a future IRC-style environment that can support large numbers of users.
The following sections introduce the main architecture of the irc++ design.
There are three type of server in irc++: user servers, relay servers and content servers.
A user server has three main rôles. First, it is the only server to which a user's client may connect directly. The other two types of server are internal to irc++ itself. Secondly, it can convert protocols, so that a telnet, mush, IRC or irc++ user can connect to a user server directly. The protocol is determined on connection, or in some cases can be changed (not all protocols need to support protocol switching, though). Thirdly, it holds digital certificates (certs) for trusted users, as described in more detail below.
The intent is that a system might run an irc++ user server automatically on startup, and any local user clients would connect to it. The user server then handles message routing and local connection policies. User servers are considered to be insecure and untrusted, so that there is no direct equivalent to IRC's per-server IRC operator; administration and control are described under The Network of Trust below.
When a user server starts up, it can use a peer-to-peer broadcast much in the style of gnutella to locate other irc++ servers, or it can be configured to contact one or more other servers directly, or a combination of both.
The intended result (this has to be measured in practice) is that user servers form a richly connected but insecure cloud. Multiple clouds are connected through relay servers.
The user server establishes secure connections to content servers using the Netscape Secure Socket Layer or a mechanism such as IPSEC (MacDonald 2000). Compression can be used. The connection between a user's IRC or irc++ client and the user server may or may not be encrypted, although if both are on the same single-user computer this is less of an issue than if the connection is over the Internet, or even over a LAN. Existing IRC clients generally don't support encrypted connections, and it is a goal of irc++ to have sufficient backwards compatibility that today's IRC users can easily switch. None the less, support for encrypted client connections is planned.
It is not necessary for every user server in a cloud to connect directly to content servers; streams can be routed from user server to user server if the connection speeds are sufficient. Since the individual streams are encrypted, this is not considered a security risk. It should be noted that the irc++ management cannot take responsibility for the security of such encryption; you should consider another medium if you are negotiating sales of armaments to warring countries. Or just don't do that.
A relay server is used to route messages. Unlike an IRC hub irc++ relay servers generally have multiple parallel connections; if a message is received multiple times, they do not need to forward all copies. The relay servers manage streams; multiple irc++ streams may be multiplexed over a single connection, and are split into packets with sequence numbers. An outgoing stream from a routing server may be marked as quiescent if it is redundant and another link is currently faster. A recipient server may send the most recent sequence number it has received on that stream, and the relay server will respond either by sending more recent packets (so that the connection is no longer quiescent) or by noting that it is out of date itself and reconfiguring its own feeds, as well as replying. The response time is measured, and the receiving server may dynamically switch between multiple relay servers, even if it was up to date.
The current design has relay servers configured manually in terms of clouds and users on behalf of whom they will forward information. Actual dynamic routing of data must be automatic, although since technologies are always imperfect, manual overrides are generally wise.
A content server is a broker: it arbitrates when a request is made that a channel be replicated, or when a user server requests a connection to a resource on behalf of a user. Content servers are also informally known as channel servers, since the channel is the IRC unit of content; a MUSH or MS user might however prefer to think of a content server as being a World Server.
Content servers do not originate content themselves, but they do allow information to be stored by sufficiently trusted users. This is described in more detail under Channels and The Content Graph below.
Channels in irc++ are multiplexed conduits for streaming data; that is, any user can talk to send an image, and the data is replicated to all users who have joined the channel.
AS in IRC, channels have a name; for compatibility, names begin with a # and are alphanumeric (#Help, #ankles, #NeWS). Channels may have labels, however, that are displayed by irc++ clients or that can be included as part of a channel topic for IRC clients. Channel labels (as with all user-supplied text) use the Unicode character set.
Channels are arranged in a hierarchy of areas; this can be represented by names separated with #: #Countries#Gymru or #Computers#WindowsingSystems#NeWS for example (the / might feel more comfortable to Unix users, and the dot to Usenet users, but # has a compatibility advantage). If an IRC client (in compatibility mode) joins #Countries#Gymru, the user server translates the request into a sequence: first a content server for the top level area # must be found; it is asked for a list of servers for #Countries, and one or more of these is queries for #Gymru. When this is found to be a content channel rather than an internal graph node, the appropriate content server is contacted and a request is made to join the channel. The content server may require credentials, may redirect the request to one or more alternate servers, or may accept the request and allow the stream to be opened to the user.
Any given channel may appear at more than one point in the hierarchy, so it is a graph rather than a tree. Two channels cannot have the same name at the same point in the hierarchy; in the prototype implementation, all channel names must be unique, but this is because of a bug.
The user server keeps track of each client's current area. An attempt to join the channel #.. or #up results in moving the current area to the parent of the current area; IRC clients are sent a list of an area when they join it, and irc++ clients will be able to receive an XML description (probably in the unified ISO Topic Map/RDF syntax when the harmonisation between those two groups is complete).
The effect of compatibility mode in mIRC (a popular IRC client for a legacy operating system) is that users can do a LIST and then double-click on listed areas to move about; double-clicking on a content channel joins the channel. The Perl prototype implements this for testing; all of the main IRC clients function correctly with it.
The user server also operates a pseudo-channel called #Server; this can be used to interact with the local server directly. This approach was taken rather than the more usual IRC approach of sending messages to a designated user such as NickServ because not all clients can handle the replies (RFC1459 requires that they be NOTICEs and not messages), and because a simpler user interface was needed for user registration. Figure 3.1 shows a sample session in the #Server channel. Note that since #Server is a fiction created by the user server, there is never more than one user in the channel, along with the Server pseudo-user. The conversation is therefore as secure as (or, as insecure as) the link between the user server and the client.
If a link is broken, the network is split into two disjoint trees;
Users in irc++ are people; a user may choose to run a bot, that is, a program that connects as a client but that speaks or reacts to input, but the bot must be associated with a user.
A new user must register before they can use irc++. It is possible to connect as a guest user, but such users have very restricted privileges.
One a user is registered, they may be sponsored; that is, one or more other users may vouch for the new user in some way. This can be done in irc++ or using HTTP over the web. When you sponsor another user, you indicate that you trust them in some way.
User can be trusted with a number of separate credentials, as follows.
Identity: that the user is who they claim to be; this is the most basic and fundamental credential, and in general any new user is likely to need several of these before they can create channels or advertise their own content;
Responsible behaviour, which includes being level-headed and acting in general fairly and in the interests of the network, including trust with authority over others;
Technical knowledge about irc++ and computing in general;
Helpfulness and lovingkindness: that is, the user's propensity to help other users, especially beginners
Architecture and Channel Design: that is, the user's ability to design channels and virtual worlds that cause surprise, delight, enjoyment or comfort, and which other people like to inhabit.
In irc++, trust is distributive: the users that a trusted user trusts are also trusted. But trust degrades over distance, so that they are less trusted. Trust also degrades over time, so that sponsorship must be renewed. This allows for people's opinions of others to change over time, and to be reflected in the network; it also allows for the case where people leave the network, as their sponsorships become less relevant and gradually fade over time.
The interested reader is referred to Raph Levien's paper on networks of trust at www.advogato.org (Levien 1999). (The uninterested reader is invited to find other entertainment!)
The trust network used by irc++ is designed to be operate without a single central server. In this regard it more closely resembles PGP than Advogato. When one user sponsors another, digital certificates are exchanged. Users gain certificates by registering with a pass phrase, or can use a public PGP key. A content server may require a particular credential before allowing access to content: for example, you might need to have several Identity credentials to say that several people trust you to be who you claim to be, before you can speak in a particular channel.
A possible vulnerability is that user servers could distribute certificates illicitly; however, the certificates must match those of the issuer, if challenged, so that this vulnerability would only allow a group of untrusted individuals to share certificates. They could do that anyway, by copying the files; since the trust of the group of individuals is no higher than the trust accorded them by people from other groups, they do not gain anything by doing that. If an evil user steals a certificate from someone trusted and claims to be that user, they still need that user's pass phrase (the private key). Digital signatures are, however, inherently insecure when stored on disk; we must do the best we can.
Actions that can be enabled by trust might include the ability to create or destroy a channel or even a whole area and all the channels within it; changing object attributes (such as the credentials needed to enter an area); banning an individual based on identity or possibly on ISP or other criteria; evicting a channel from an area, or even creating new objects and properties.
A user may choose to be
Where IRC has single-character user and channel modes, irc++ has objects with named properties.
An object is a collection of property/value pairs. Objects have a type, such as channel, area, credential or old boot. Objects are stored by content servers, and can be distributed to secondary content servers for robustness (Emmerich 2000). Objects also have an isa property, and can inherit properties. There are no plans for multiple inheritance at this time.
Object properties consist of a triplet: name, type and value. Any user can (with the necessary credentials) retrieve the value of a property, assign a new value to any property, and copy properties from one object to another. Some examples might make this clearer.
set #Gymru topic "Beth ydy goat yn Gymraeg?"
this corresponds to the IRC TOPIC command.
set #Gymru permissions.maxusers 20
might change the maximum number of people allowed to enter that channel,
and would correspond to the IRC command MODE #Gymru +l 20.
The entry property of a channel lets people join the channel. By default, only the founder can join, but since this isn't useful, the founder is told how to change it on joining. This gives the founder of a channel time to set it up before others enter.
set #Bodelwyddan entry.credentials credential:canjoin(#Gymru)
might be used to say that people can only join #Bodelwyddan (a castle in
Wales) if they can also join #Gymru.
Note that for these examples to work, you must either have joined the channel, or have your current area be one which contains the channel; otherwise, you would need to use the fully qualified name, #Countries#UK#Gymru for example.
The combination of areas and the ability to require that you be in one channel before joining another means that you can implement a foyer channel; this lets people in the inner channel vet people who wish to enter, without the disruption of a knock system.
It is possible to arrange for an object to be transferred from one channel to another, or for a user to be given ownership of an object. Objects are initially owned by the identity that created them; they can be transferred and exchanged, subject again to credentials. This is the start of a MUSH or MOO-like environment: the idea is that a channel owner can create an object such as a key that opens a door. Scripting to add commands, however, might involve a script server; this has not been thought out in detail.
The main goals of irc++ are to provide a vastly scalable infrastructure, so that over a million users can be online at once, and to support for multimedia information exchange in real time. This places constraints on authority and accountability, on resource discovery and on network routing; each of these is described in the sections that follow.
Traditionally, IRC uses a model that binds authority to the people who run IRC servers (admins) and at a smaller scale to people who run channels (channel operators). Admins can appoint IRC operators who can ban troublesome users, close channels, and generally help to keep things running smoothly.
There are a number of problems with this setup, the greatest of which is that as an IRC network grows the burden on the staff increases; larger networks create additional layers of staff, but this means that the network becomes impersonal. A self-managing system is clearly advantageous here, although it is also untested.
An IRC network with 50,000 active users online (very large indeed for an IRC network) will typically have over 10,000 active channels. The names are global, and are presented in a list. Worse, the IRC protocol allows for transfer of the entire list, or of the information for a single channel, not for, for example, the first 100 entries, then the next 100. Figure 5.19 on pages 23 to 48,301 shows a sample listing for a network with 100,000 users [deleted by editors]. Clearly, finding a channel that is of interest can be a major undertaking, and the channel hierarchy is a major step forward here.
Because IRC networks replicate information across all servers, a message is broadcast whenever a user joins or leaves a channel. This means that bandwidth requirements grow as a network grows. The channel hierarchy means that a user server does not need to have any information about areas not currently in use on that server, except to know how to find them.
It is also hoped that irc++ is sufficiently flexible that many smaller existing IRC networks will want to move to it, so that there will be more resources (both channels and users) to find.
It should be noted, however, that where IRC keeps running totals of users and can instantly tell if a given user is online with a given nick, irc++ keeps no such information. A search for a user involves an attempt to contact their primary user server, and in this regard irc++ is more like an Instant Messenger service. Groups of users could of course choose to rendezvous by joining a particular channel solely for the purpose of being visible; there are some large IRC channels used in this way currently.
Allowing multiple redundant links considerably reduces network traffic for non-leaf servers. However, the current IRC protocol assumes a minimum spanning tree. Figure 4.1 shows a possible IRC network in which there are six servers, imaginatively named A, B, C, D, E and F. Each of these is connected in some way to the Internet via a router; the link between a server and the router might be Ethernet, or it might include an ISDN or T1 connection before higher bandwidth is reached. Suppose there is one user on each server. If a user sends a private message to every other user, the server must send five separate messages. If they all do it, thirty messages are sent; the figure shows how many messages go over each segment of the network. A sample path for a message from E to C is also shown.
Most IRC routing diagrams neglect to consider the fact that it is the IRC server, i.e. ircd, that performs the routing, not the Internet router. The message from A to C must enter D, be rerouted to B and leave again, and thus contributes twice to the load on D's connection. If we add a couple of extra connections, however, from A to B and D to C, we can see that the peak loads at D and B are reduced significantly. Since irc++ has every user server connect directly either to a channel server or to one or more designated relay servers, not only is robustness increased, but peak bandwidth requirements are reduced.
Figure 4.2 Adding links reduce bandwidth needs.
With extra links and redundant connections, packets may be duplicated for reliability. In order to avoid duplicate or lost information, sequenced packets must be used. Despite the need for adding sequence numbers, the new format uses considerably less space, and is also faster to parse. At the time of writing, the irc++ packet formats are not completely fixed; rather than give out-of-date details here, the reader is referred to the web page (Quin 2000).
Several references have been made to IRC clients; at the time of writing there are no irc++ clients. All additional functionality is accessible to IRC users, although it is not always as convenient as it might be if the client had direct irc++ support. The most obvious differences include the channel hierarchy, properties, the lack of the notify facility (although this may be corrected in the future) and the lack of netsplits.
There is a prototype server written in Perl to test the user interface issues; this has met with some interest, and the channel hierarchy has been shown to be feasible.
This document is to be considered as a discussion piece; the implementation is shifting, and many features will undoubtedly be simplified. None the less, readers are invited to contribute ideas and code to the project.
As it stands, the irc++ design shows that a more robust system can be built, and that such a system need not represent a dramatic increase in bandwidth usage. Furthermore, user interface changes and management issues such as the content hierarchy/graph and certificates of trust will permit irc++ to scale to a much greater size than is possible with IRC today, yet with fewer disruptions.