Re: Starlink terminal data acquisition for network engineers

[Tim Nelson <tnelson () sangoma com>]

Thanks Eric, this is great information. I just installed my Starlink
"dishymcdishface" this weekend and it's humming along nicely, even in
the -25F (-51F with windchill) temps here in northern MN! I've got a
bunch of general monitoring of the link using a few Docker + grafana
widgets [1], but hadn't looked into direct metrics gathering from the
terminal yet. This will surely get me on that path :)

--Tim

[1] https://hub.docker.com/r/qlustor/speedtest_ookla-to-influxdb




Tim Nelson
Network Engineer / Cloud Services
Phone: (218)727-4332 x4501 / Fax: (866)716-0229
5019 Airport Road, Hermantown, MN 55811
Check us out at www.sangoma.com




On Sat, Feb 6, 2021 at 4:06 AM Eric Kuhnke <eric.kuhnke () gmail com> wrote:
> I thought about posting this to only NANOG, but since a great concentration of beta testers of a technical/network 
> engineering inclination are located in the Pacific NW, decided to also include the SIX chat list.
>
> You may have seen the Starlink android or ios consumer-friendly app, which displays network traffic, uptime/downtime, 
> and other link stats. I believe this to be polled directly from the antenna unit itself over grpc.
>
> The beta antennas are always 192.168.100.1. If you are using your own router with the starlink beta system, in 
> addition to its WAN interface being an ordinary DHCP client in cgnat IP space, you'll need to manually give it an 
> address in that /24 and set up routing to reach the .1 IP as needed.
>
> reference: https://github.com/sparky8512/starlink-grpc-tools
>
> reference: https://github.com/fullstorydev/grpcui
>
> you'll need a fairly normal Linux or BSD box with:
>
> git
> go
> python3
> pip
>
> use pip to install grpcio and grpcio-tools
>
> install grpcurl: https://github.com/fullstorydev/grpcurl
>
> do a git clone of the starlink-grpc-tools url above, also take a look at its readme info
>
> get the dish's protoset file and write it to new file dish.protoset , this is an index of all data that can be polled
> cd /home/eric/starlink-grpc-tools
> /home/eric/go/bin/grpcurl -plaintext \
> -protoset-out dish.protoset \
> 192.168.100.1:9200 \
> describe SpaceX.API.Device.Device
>
> /home/eric/go/bin/grpcurl \
> -plaintext \
> -d {\"get_history\":{}} \
> 192.168.100.1:9200 \
> SpaceX.API.Device.Device/Handle | python parseJsonHistory.py
>
> output of the above looks like this:
> 2021-02-06T08:15:56,3600,19.92034332,14,2,0.3125,0,0,0.0,0
>
> full CSV header for the above:
> datetimestamp_utc,samples,total_ping_drop,count_full_ping_drop,count_obstructed,total_obstructed_ping_drop,count_full_obstructed_ping_drop,count_unscheduled,total_unscheduled_ping_drop,count_full_unscheduled_ping_drop
>
> since we are able to acquire the above in a comma-delimited csv format, it's fairly easy to write a script storing 
> the integers from any one of those particular columns into a mariadb db, sqlite, influxdb, or whatever.
>
> the following will output about 3.8MB of text for the full history (I believe this to be the full copy of the ring 
> buffer stored in RAM for the terminal's statistics) , pipe it into a text file if you want to manually look at it.
>
> /home/eric/go/bin/grpcurl -plaintext -d {\"get_history\":{}} 192.168.100.1:9200 SpaceX.API.Device.Device/Handle
>
>
> same as the above but human readable output
>
> /home/eric/go/bin/grpcurl \
> -plaintext \
> -d {\"get_history\":{}} \
> 192.168.100.1:9200 \
> SpaceX.API.Device.Device/Handle | python parseJsonHistory.py -v
>
> current counter:       299673
> All samples:           43200
> Valid samples:         43200
> Parsed samples:        3600
> Total ping drop:       20.03700998
> Count of drop == 1:    14
> Obstructed:            2
> Obstructed ping drop:  0.3125
> Obstructed drop == 1:  0
> Unscheduled:           0
> Unscheduled ping drop: 0.0
> Unscheduled drop == 1: 0
>
>
> see the get_history_notes.txt file for more info
>
>
> SOME EXAMPLE QUERIES
> these should match with what the json query is in the grpc GUI
> # get status
> /home/eric/go/bin/grpcurl \
> -plaintext \
> -d {\"getStatus\":{}} \
> 192.168.100.1:9200 \
> SpaceX.API.Device.Device/Handle
>
> # get device info
> /home/eric/go/bin/grpcurl \
> -plaintext \
> -d {\"getDeviceInfo\":{}} \
> 192.168.100.1:9200 \
> SpaceX.API.Device.Device/Handle
>
> # get history, this outputs a huge amount of data
> /home/eric/go/bin/grpcurl \
> -plaintext \
> -d {\"getHistory\":{}} \
> 192.168.100.1:9200 \
> SpaceX.API.Device.Device/Handle
>
>
> The following is copied/pasted from my notes file on things we can acquire, and then use a tiny shell script with 
> awk, sed, regex, whatever as needed to trim out just the numbers, and put them somewhere else for polling by snmp 
> extend
>
> /home/eric/go/bin/grpcurl \
> -plaintext \
> -d {\"getStatus\":{}} \
> 192.168.100.1:9200 \
> SpaceX.API.Device.Device/Handle
>
> notes on what's what:
> figures we care about to parse out and turn into just the integers
> snr can never be higher than 9
> fractionobstructed appears to be the percentage of the time that the view is obstructed, as long as the view
> remains unobstructed, this number appears to slowly decrement over time
> validS is valid seconds? i think
> the S is almost likely always Seconds of time
>
> "uptimeS": "304439"
> "snr": 9,
> "fractionObstructed": 0.0013524292,
> "validS": 61815.74,
> "last24hObstructedS": 53
> "downlinkThroughputBps": 47915.73,
> "uplinkThroughputBps": 34980.496,
> "popPingLatencyMs": 29.266666
>
>
> example of running the command above twice, the second time a few minutes after the first,
> to see that fractionObstructed does decrement itself over time
> first run: 0.0013524292
> second run: 0.0013467998