Joost down for the count a second time

There is an adage that say that “Lightning never strikes twice in the same place”. Joost might just have been hit a second time in a day!

Joost reported yesterday that an SSL certificate expiry issue prevented all 0.9.1 beta client from connecting to the network. Joost apologized in an email instructing beta testers to download a copy of beta client 0.9.2 to resolve the SSL certificate issue.

For a few hours all appeared to be good. But later in the day all network connections came to a complete halt. No one appear to know what is going on at this point.

Looking at a network trace of the communication between the client and the back end servers it look like a wrong IP address is sent to the client during the initialization phase. Instead of receiving a publicly reachable address on the internet the client get one in the non routable space (192.168.165.x/24). The client try to connect to it and obviously will not be able to… hence the long period of inactivity until the client stop trying an report an error.

The served IP address appear to be a neighbor of the ones affixed to Joost servers in one of their data center as can be seen in this picture.

What does this mean? Someone made a BIG boo boo with the back end server configuration.

Joost really needed this to happen after the SSL certificate incident.

Someone job must be on the line right now ;-(

Let hope this get resolve shortly.

Update:

At around 21h20 GMT – 4 the wrong IP stopped being served to the clients. At this point there is no IP given back to connect to and the client quickly error out… in a similar manner as yesterday.

Update 2:

At 22:17 GMT – 4 Joost is back online! Hourray!

Digg this story