[Hands0n]

I am disturbed. Very disturbed. In fact, I'm so disturbed that I am actually quite worried. Vodafone's mobile broadband network has all of a sudden got rather sucky.
1119

Now this is not good at all. Previously I would easily be able to get multi-megabit downlink and somewhere around 300kbps to 500kbps uplink speeds. Yet all of a sudden the speeds have dropped, particularly the uplink.

I had thought this was only a problem back at the office, but the above result is taken from my home which is approximately 150m from the Vodafone 3G mast.

Not good at all. Especially considering when I am getting significantly better performance from a local Three 3G mast which is almost the exact same distance from my home, both with clear line of sight!

So what gives, Vodafone?

[Ben]

It's not just you, either! I've been struggling with mine!

In fact, one of my colleagues has my Pro at the moment and, despite having 3G signal, they say the speeds are very hit and miss - often miss.

I've taken to using my MiFi on Three. Just can't fault it at the moment, not relatively speaking anyway.

govenden has my older Vodafone dongle, that one always had really good reception (though it was huge, lol) so I'll try and remember to ask how that's going tomorrow. Apparently it survived iPlayer and the IT Crowd, so it must be working relatively well ;)

[Hands0n]

Yes, this is not a signal strength issue. At home I have five bars of the stuff. At work the signal is 3 to 5 bars.

At all times I have a 3G signal, and voice quality is perfect. This is completely a data quality issue that is impacting speed. From some very early testing it looks like jitter on the uplink. For this to be so widely manifest it must be in the core mobile network. Proving that as an outsider will be very difficult indeed.

[hecatae]

so what happens when you try www.pingtest.net ?

[Hands0n]

So, taking the analysis a bit further ...

A pingtest run using ADSL broadband gives reasonably expected results as so
1120

This morning's results with the Vodafone mobile broadband service continue to be disturbing
1122

The packet loss is what makes performance disruptive and should not be there at all, but accepting that it can happen occasionally. The trouble is that at the moment Vodafone's lossiness seems to be ever present and highly variable. Look at this next result taken a minute later - 51% packet loss! That is completely unacceptable.
1123

Basically, no network should be showing such highly variable results - from zero to 51% packet loss across tests.

From time to time the mobile data does work well and performant - well it would do if the packet loss is zero (see the other attachment below).

I'll forward the link to this thread into the Vodafone eForum team and hopefully they can prompt an investigation by their network engineers, if they don't already know (I can't believe that they wouldn't know of a problem on this scale - but wouldn't expect them to publicly admit it).

So, watch and wait people. But right now, if you are dependent upon your mobile broadband, you may want to consider an alternative, or at least plan for the worst.

[Ben]

Argh, that packet loss would explain a lot :(

[miffed]

The iPhone effect IMO

[hecatae]

is there a difference in equipment that maxes at 3.6Mbps and equipment at 7.2Mbps?

[Hands0n]

@miffed - I doubt it very much. It would take an extraordinary level of congestion to cause jitter like that. In all my days networking it is only on very low-speed circuits with high congestion rates that I've seen the effect. Now, I could imagine this on isolated cells with low-speed backhaul circuits in densely populated areas with few cells. But this appears to be fairly widespread and certainly where I am the density of population is relatively low. The low throughput is a direct consequence of the jitter and packet loss.

@hectae - the difference would be in the overall bandwidth capability of the specific cell. It would then carry fewer concurrent users at higher speeds, and I would expect to see speeds being low, the bandwidth being distributed over the concurrent users statistically. The packet loss and jitter seen above suggests complete saturation of the facility - but again, I would not expect to see it widespread across large geographic areas. To be so it would suggest that the actual core network is overloaded. Which would imply that Vodafone's core data network is significantly under-invested. Could that really be the case?? If true it would be a body blow to Vodafone's reputation of being a premier mobile data service provider to Enterprise.

Given the suddenness, I wonder if Vodafone have introduced some packet shaping into their network and have got the settings wrong in some way! I would expect to see the networks introduce packet shaping in order to be able to sell tiered tariffs which must be on their way soon. They'll be wanting to make money out of the mobile data scene.

[Hands0n]

Would any of you reading this who have similar performance problems on Vodafone's mobile data network please visit their eForum and pop feedback on there also. The direct link to the thread I started is --> http://forum.vodafone.co.uk/topic/70698-mobile-data-severe-performance-degradation/

Already one of the eForum Mods has responded with interesting news. It suggests that Vodafone's network engineers already are aware of problems


I'd like to keep this thread open and look out for any additional queries relating to the same issues, I am aware that engineers are working on certain network issues and would give this a few more days to clear up.

If the issue persists after this, or if you notice no improvement please post back on the thread so we can perform an additional follow up.

One could speculate what "... certain network issues ..." are but what is particularly satisfying is that they already know that something is awry and are taking steps to resolve whatever it is. :) And I suppose that is what Vodafone are all about - they fix stuff when it goes bang! :D

[DBMandrake]

The poor upstream speed (110kbit) is the clue here - I'll bet anything that HSUPA is broken or turned off - forcing all customers back to plain 3G for upstream bandwidth, which will cause huge upstream bottlenecking, high latency and jitter.

Why would this happen ? Well, AT&T in the US just had something very similar happen a few weeks ago after the iPhone 4 release - the first iPhone that supports HSUPA. Initially people were seeing 1Mbit or more upstream but then it dropped to 50-100kbit for a few weeks. AT&T fixed it after about 3 weeks and identified it as a "software issue" with their UMTS base station equipment.

My guess is either it was a software update they rolled out that broke HSUPA support (so they had to disable it until they could roll back the software or get a further update to fix it) or the sudden influx of data hungry HSUPA capable iPhones exposed a bug or issue in the existing base station software. I bet Vodafone are using some of the same brand base station equipment as AT&T....(there are only about 3/4 brands of equipment in use for UMTS networks anyway)

[Ben]

It'd be interesting if this is iPhone related, especially after Vodafone's comments when O2 were struggling so badly. There's no doubt in my mind that iPhones must place an extraordinary strain on networks not previously used to them... lots of connections, not necessarily transmitting a lot of data - short, bursty, expecting things fast.

[Hands0n]

So with some reasonably extensive testing both in London and at home in North Kent with identical results the evidence is that the issue is firmly within Vodafone's network.

Using Pingtest I managed to achieve some 51% packet loss at worst, 4% at best ... and that is going to ruin any kind of performance.

The Vodafone mods asked me to post on the eForum which I did and there is a tacit admission that their engineers area already working on the fault last week. So it is not unknown. But here we are, Thursday, a week on almost, and no change. The fault is as bad as ever.

I work for a telco and it would be unthinkable to have such a bad network degradation issue on the go for so long. I just have to wonder if there is any regulation of mobile network operators for data performance, availability and quality. It seems not to be the case, historically.

[miffed]

Sorry , can't add anything helpful , but I just wanted to say , after reading recent postings regarding Vodafone ...

... this must be how Noah felt when he noticed his neighbours rushing around to get their washing in. ;)

[Hands0n]

... this must be how Noah felt when he noticed his neighbours rushing around to get their washing in. ;)



ROTFL, PMSL and any other appropriate (or inappropriate) acronyms.

Really, it is dreadful right now, but also it is extraordinary in my own experience of Vodafone's mobile data network. Unprecedented too.

[3GScottishUser]

Vodafone Mobile Broadband has been very good for me lately. It's been highly reliable on the new Galaxy S in the UK and in Turkey it was top notch.... at £3.50 for unlimited data at 7.2Mbs for a month. :)

[Hands0n]

So how are we doing almost one month on since I brought this up on here and on Vodafone's forum?

The answer, it seems, is not very good. Lets see the Pingtest results for today

1124

The next test run was 26% packet loss - but it wouldn't complete and I had to cancel it!

After a gap of several minutes the next test threw up 71% packet loss - with a final result
1125

Thats a "best of three" with only two completing. So just to be fair I ran a fourth test several minutes later. That, too, would not complete!

Speedtest results were better than they have been for some time
1126

But one can't help feel that they would have been significantly higher [speeds] if the underlying TCP/IP protocol were not having to deal with all of that packet loss and jitter.

It isn't great, still.

[Ben]

That's still very poor. I wonder if we'll ever see Vodafone back in the 120ms range for ping, and sorting out these apparently horrific jitter issues? (I haven't experienced the packet loss).

[Hands0n]

One would hope that Vodafone & O2 can resolve this somehow. I'm fairly convinced that this is related to their network share, given the timing. Likely they've made a join somewhere and its all gone a bit loopdy loo (thats a technical term :D )