High packet loss on the Three and Eir 4G networks

[Seán]

Since around the start of August '20, I am experiencing very high packet loss on Three 4G in the evenings, particularly off the Ox Mountains mast in Co. Sligo (cell ID 828423) which I've a pair of antennae pointed at with clear line of sight:

   

The packet loss is low before 6pm (0-1%) and peaks around 9-11pm, often exceeding 10%. It occurs intermittently, so could be 0% one minute and over 20% the next minute like the above example.  From testing with the Net Analyzer app on my phone in several areas, it appears to mainly affect rural 4G masts, at least in various areas I tested across south west Donegal.  The following test by the app was in Killybegs, Saturday evening close to 7pm:

   

Each line represents a ping reply time, one per second.  Missing lines represent lost packets.  The above graph also shows an example of it being fine one minute and dropping packets the next. 

For anyone wondering if their recent performance issues are caused by packet loss, you can run a ping test on Windows as follows: 

1. Right-click the Start button.
   
2. Click 'Windows PowerShell' or 'Command Prompt'. 
   
3. Type: ping -t 8.8.8.8
   
4. Allow this to run for a few minutes, then press Ctrl + 'C' to stop and show the packet loss %.
   

I will update this post once the packet loss has been fixed.  However, based on the last time I had high packet loss a few years ago, it took Three several months to fix, so for anyone experiencing this issue, the only suggestion I have is to switch to 3G during peak time, which does not appear to be affected.

Update - 10th August:

While waiting in the Kingsbridge private hospital car park about 6:30pm, I ran a ping test over Three 4G+ there.  The packet loss was around 12% as shown below and similar along the Sligo main street.   

   

I tested a few other areas on the way home (as a passenger) and had similar packet loss in a few other areas such as outside Bundoran.  There are a few areas that seem unaffected, for example, I had no packet loss in Donegal town and the same with Dunkineely, so those 4G masts are probably fed off a different backhaul. 

I also ran a few more tests on 3G and besides ping spikes (to be expected on 3G), there was no packet loss, even with the weak reception at the Top filling station outside Killybegs:

   

However, once I entered Killybegs and on 4G+, the packet loss was there.  Surprisingly, it was very high even earlier after 4pm:

   

[Seán]

In addition to high packet loss, my Three data connection kept restarting every few minutes, causing video calls to cut for a few seconds each time. 

It came to the point where I tried using a GoMo SIM in my phone as a hotspot, only to discover high packet loss also on Eir's 4G network. 

Packet loss on the Eir 4G network ~10pm:
   

Like Three, when I switched to 3G mode, the connection was fine with not a single lost packet after 5 minutes:
   

This means that the backhaul issue is not limited to Three, but instead likely with the infrastructure sharing between Eir and Three.

Vodafone does not have any 4G signal in my area and no longer have an active Vodafone SIM, so am unable to test their 4G network for packet loss at this time.

[Seán]

Just to update, I was in Letterkenny today and just like other areas, there is high packet loss both there and in Ballybofey.  While waiting in the car park at TK Maxx in Letterkenny, I left the ping test run for 5 minutes and this is the result:

   

I tried a few other areas and had similar packet loss.  Ballybofey was much the same on 4G+.

While at the picnic area in Barnesmore Gap (midway between Ballybofey and Donegal), I left my phone pinging away.  Whatever backhaul that mast connects to, it has no packet loss at all.  I reran the test a few times just to rule out a fluke.  The following is one of the tests run over 5 minutes:

   

This is not to say that the 4G network was fine there either.  Despite the time around 4:30pm, it was crawling.  I'm surprised the mast there only operates on 4G band 20 considering how congested it was based on getting speed tests of under 0.5Mbps at the time and this is not even peak time!

   

My home connection remains terrible after 9pm, hitting 18% packet loss around 10pm this evening.  The following was around 11:30pm over my Three mobile broadband Wi-Fi connection:

   

[Seán]

This issue appears to have been fixed.  So far this evening, I've had a packet loss of less than 1 per 1000, i.e. <0.1%.  One test with over 2000 pings did not have a single lost packet:

   

I'll do some further testing over the coming days including on the Eir network to rule out a fluke.

[Seán]

So far I have not had any further issue with elevated packet loss on the Three network.

I finally got around to testing a GoMo SIM and packet loss remains on its network, at least with both bands 20 and the temporary band 28 that I pick up here:
   

[Patrick]

   
Just got this Sean, first time doing it,  on 3, Co. Cork

[Seán]

That result including the latency are good, roughly the same figures on my Three connection at the moment.

The time figures usually go high with traffic on the network, particularly while uploading, however, the packet lost percentage should ideally remain zero.

[Patrick]

   
Neighbour sent me this Sean, no aerial. On 3, same as me, but I hardly get any drop, about 70 all day. Massive difference in speeds in evening. Could peak usage cause this much of a drop? I'm nearby with a rural mast, but they might be connected to a town mast as it's nearer? I'm thinking town mast is busy in evening and rural one not? (Imagine has done the area, so reckon takes pressure off local masts     ) 

Also is it better to start new thread or tack onto existing if q is unrelated?

[Seán]

That certainly looks like a very congested mast, very likely a lot there depending on Three or using their phones for streaming. I suggest they try moving the router to the opposite end of the house to the town to see if it will pick up the other mast. They would certainly benefit with an outdoor directional antenna facing the rural mast.