Re: ARM build slaves (tcpdump mirror in Germany)

[Denis Ovsienko via tcpdump-workers <tcpdump-workers () lists tcpdump org>]

> --- Begin Message ---
>
>
> From: Denis Ovsienko <denis () ovsienko info>
>
> Date: Tue, 23 Mar 2021 00:35:06 +0000
>
> On Mon, 22 Mar 2021 19:00:31 +0100
> Harald Welte <laforge () gnumonks org> wrote:
>
> > Dear Denis,
> >
> > On Sun, Mar 21, 2021 at 11:08:44PM +0000, Denis Ovsienko via
> > tcpdump-workers wrote:
> > > Thank you for the offer. For the operating systems specifics please
> > > see below.  
> >
> > thanks for the details.
> >
> > In general, any of those can be hosted by sysmocom.  Should we just
> > order those four raspi's?  We're familiar with Raspbian and Debian as
> > well as most Linux systems in general, but not very much so with the
> > various BSDs.
>
> Hello Harald.
>
> That's great, three worker types are ready to go anytime soon, just
> decide which you are taking and when.
>
> * linux-aarch64 -- the simplest one. RPI4B would be the best for it as
>   far as Pi models go. The worker runs fine on as little as 1GB RAM,
>   but the smallest RPI4B you can buy today seems to be 2GB. Raspbian or
>   Debian would be fine if you prefer to install Linux yourself. The
>   Raspbian I have tested to work was pre-installed, so exact setup
>   details were unknown. The Debian 11 snapshot I have confirmed working
>   comes from here: https://raspi.debian.net/tested-images/
>
> * freebsd-aarch64 is ready to go, also RPI4B. It will be necessary to
>   re-flash the SD card after FreeBSD 13.0 release is out and confirmed
>   working (that is, in 1-2 weeks).
>
> * netbsd-aarch64 is ready to go, the right Pi model for it would be
>   RPI3B+.
>
> I presume you have already figured cooling out for RPI4, please mind
> RPI3 generates more heat, so it would be best to run it with both
> heatsinks and a fan to keep it under the thermal throttling threshold.
> Mine was throttling under load until I got this detail right.
>
> > Inbound Ethernet/IP access (via port forwarding from a static IP) is
> > possible; we can also set up remote serial console via TTL/UART .
> > Remote HDMI/keyboard is unfortunately not an option.
> >
> > For those BSDs, it might make sense if 
> > 1) we purchase the hardware
> > 2) somebody has bootable images to simply provide those
> > 3) we 'dd' the images to the SD card / USB drive, make sure they get
> > a static IP on a "DMZ" VLAN and set up the port-forward and, as
> > needed, TTL UART access to the serial console via a different machine.
>
> You will receive ready to run SD card images.
>
> > > One important thing to keep in mind is that the workers may end up
> > > executing arbitrary commands when building a pull request
> > > automatically. So it is best to keep them firewalled from internal
> > > networks and not to keep any other important workload/data on the
> > > same host.   
> >
> > In terms of network: Sure, we do this the same for the other jenkins
> > slaves we operate.  They are in their own subnet/VLAN which only
> > permits public IP access (via NAT).
> >
> > > It might be possible to use some virtualisation means to isolate
> > > the workers from the rest of the host, but please do not count on
> > > that anytime soon unless you can do it yourself.  
> >
> > I think at the low cost and low pwoer consumption of raspi's, I would
> > have simply assumed to use one dedicated unit for each of those
> > builders...
> >
> > For Linux, we have positive experience with putting build slaves in
> > lxc containers on top of Raspbian (e.g. Debian containers on
> > Raspbian).  So if it is about multiple distributions that can run on
> > the same kernel version (and the overall load is low enough), that is
> > one option.  For the BSDs, I would suggest to keep it simple and on
> > dedicated hardware?
>
> One physical Pi per one distinct worker indeed keeps things simple at
> this scale, that's how it is done now. A need to test more than one
> Linux distribution is unlikely.
>
> > > Let me know which ARM workers look feasible to you. If you can host
> > > PowerPC workers instead or in addition to that, it would help too.  
> >
> > Depends on the hardware.  If it's reasonably small and either
> > inexpensive or can be provided, we  can certainly also look at hosting
> > that.
> >
> > I only have an ancient "Titanium Powerbook G4" and a similarly old
> > MPC7400 based Apple machine in my retrocomputing collection, and I
> > would certainly not recommend using any of those for power
> > consumption and processing power reasons...
>
> Re PowerPC, let's omit it for clarity then.
>
> > btw: I'm not sure if qemu full system emulation of e.g. ppc on a
> > x86_64 hardware would be an option, though.  I think
> > openbuildservice.org is doing that a lot for building packages on
> > less popular architectures.
>
> QEMU was very useful for the NetBSD setup. NetBSD for some reason did
> not provide binary packages for 9.1/aarch64, and heavy non-default
> packages (LLVM, Clang, GCC 10) just do not compile on 1GB RAM of RPI3B
> (NetBSD release does not run on RPI4B), so the only way to compile
> these was in a QEMU VM with more RAM.
>
> That said, on a Linux host with i7-3770 CPU the QEMU guest measured at
> 64% core-to-core CPU performance of an RPI3B. So after the initial
> setup a hardware Pi does a better job.
>
> -- 
>     Denis Ovsienko
>
> --- End Message ---
_______________________________________________
tcpdump-workers mailing list
tcpdump-workers () lists tcpdump org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers