Alex Rivera | Logout

Scaling Tigase XMPP server on Amazon EC2

Asked 2011-12-29T16:12:31.710
10

Does anyone have an experience running clustered Tigase XMPP servers on Amazon's EC2, primarily I wish to know about anything that might trip me up that is non-obvious. (For example apparently running Ejabberd on EC2 can cause issues due to Mnesia.)

Or if you have any general advice to installing and running Tigase on Ubuntu.


Extra information:

The system I’m developing uses XMPP just to communicate (in near real-time) between a mobile app and the server(s).

The number of users will initially be small, but hopefully will grow. This is why the system needs to be scalable. Presumably for a just a few thousand users you wouldn’t need a cc1.4xlarge EC2 instance? (Otherwise this is going to be very expensive to run!)

I plan on using a MySQL database hosted in Amazon RDS for the XMPP server database.

I also plan on creating an external XMPP component written in Python, using SleekXMPP. It will be this external component that does all the ‘work’ of the server, as the application I’m making is quite different from instant messaging. For this part I have not worked out how to connect an external XMPP component written in Python to a Tigase server. The documentation seems to suggest that components are written specifically for Tigase - and not for a general XMPP server, using XEP-0114: Jabber Component Protocol, as I expected.

With this extra information, if you can think of anything else I should know about I’d be glad to know.

Thank you :)

amazon-ec2 xmpp tigase
Edit
Report

1 Answer

26

I have lots of experience. I think there is a load of non-obvious problems. Like the only reliable instance to run application like Tigase is cc1.4xlarge. Others cause problems with CPU availability and this is just a lottery whether you are lucky enough to run your service on a server which is not busy with others people work.

Also you need an instance with the highest possible I/O to make sure it can cope with network traffic. The high I/O applies especially to database instance.

Not sure if this is obvious or not, but there is this problem with hostnames on EC2, every time you start instance the hostname changes and IP address changes. Tigase cluster is quite sensitive to hostnames. There is a way to force/change the hostname for the instance, so this might be a way around the problem.

Of course I am talking about a cluster for millions of online users and really high traffic 100k XMPP packets per second or more. Generally for large installation it is way cheaper and more efficient to have a dedicated servers.

Generally Tigase runs very well on Amazon EC2 but you really need the latest SVN code as it has lots of optimizations added especially after tests on the cloud. If you provide some more details about your service I may have some more suggestions.

More comments:

If it comes to costs, a dedicated server is always cheaper option for constantly running service. Unless you plan to switch servers on/off on hourly basis I would recommend going for some dedicated service. Costs are lower and performance is way more predictable.

However, if you really want/need to stick to Amazon EC2 let me give you some concrete numbers, below is a list of instances and how many online users the cluster was able to reliably handle:

  • 5*cc1.4xlarge - 1mln 700k online users
  • 1*c1.xlarge - 118k online users
  • 2*c1.xlarge - 127k online users
  • 2*m2.4xlarge (with 5GB RAM for Tig
answered 2011-12-29T19:38:19.767

Your Answer