UUID: 000011302025 | Documentation ArchThinkCluster

————————————————————————————————————————————————————-

 Main Notes/TODO

  What have I done so far?

  I started by setting up the ProxyDHCP PXE boot system. Then I brought the NAS online and made it stable, complete with auto-restart and auto-backup. After that, I installed SLURM and instaled/set up the MUNGE key and the SLURM control daemon but i dont know if SLURM is fully up and ready yet. I had to set up a repo and do a lot to make that work. i also built the ACNST (Arch Cluster Network Scan Tool).   At some point I have to set up the “custom ISO” and thats probobley the next thing on the list i need to do i should also start building backups for my local mirrors but i dont need to do that for some time

 ##// TODO (do in order)

    make the custom iso for the worker node i dont think we need to do much for this we more or less just need an iso of arch linux the self boots in the same way evry time with a few extra tools i think. make sure slurm is indeed working. boot first slave node

————————————————————————————————————————————————————-

  System Specifications Notes

 # LAB Setup :

  LAB1: work computer, LAB3: mobile work terminal 1, LAB4: mobile work terminal 2. Xfinity home router, network switch 1: TP-link TL-SG108E switch.

 # Cluster Setup :

  5x Lenovo ThinkCentre Micro Desktops 16GB DDR4 RAM 256GB SSD Drive AMD Processor Model Name M625, Network Switch 2: STEAMEMO 16 Port Gigabit PoE,                                                                CENMATE Aluminum 6 Bay Hard Drive Enclosure, 3× 2TB hard drives

 # Network setup :

  I do not know proper technical terms, so bear with me.

  The data flow starts at the router this is the section I call the Home Network, where all of the home devices live, connected directly to the Xfinity router modem through WiFi. Past this,                       we have Network Switch 1. Network Switch 1 is connected directly to the Xfinity router modem via CAT9 Ethernet cable. LAB1 is connected to Network Switch 1 with a CAT9 Ethernet cable. Past Net Switch 1,       we have Network Switch 2, which is directly connected to Net Switch 1 via CAT9 Ethernet cable. Network Switch 2 is the hub switch for the cluster,                                                               all the cluster machines are connected to Network Switch 2 with a CAT9 Ethernet cable such as thinkcenter001 the master node and NAS1 the NAS.

#    [wifi home network]      [LAB1, 3D printers, ect]    [Cluster, master nodes, slave nodes, NAS1]

#           \                                |                                   |      

#    [router modem]<——CAT9——>[network switch 1]<——CAT9——>[network switch 2]

————————————————————————————————————————————————————-

 Network Notes/TODO

 # Top Notes :

 I did a ton of shit on the cluster, and I honestly don’t know what I did. I set up SLURM and the MUNGE keys, and some scripts that bring the proxy DHCP server back online after a reboot.                       So with that being said, I am just going to go over each thing I did and figure it out.

## //TOO-DO

   Bugs. we got bugs.

   bug list :

   # we still have systemd-networkd trying to do things we need to remove it

—————————————————————————————————-

 The PXE boot system                            

 The PXE boot system is primarily composed of two services. The first one that I had to build from scratch or set up by hand is the ProxyDHCP + TFTP server run as the dnsmasq service.

 This service is what is handing out the PXE boot info and the iPXE (a modern open-source replacement for PXE) binary over TFTP (Trivial File Transfer Protocol).

 It does not hand out any IPs and keeps no leases, and the router hosts a full DHCP server. The second service is PXE-HTTP.Service, this is the actual repository.

 The packages the TFTP server is transferring to the booting node.

 # Notes –

 i feel this system was made blide we just kept building till a goal was met

—

 # What is the service dnsmasp.service

 The service dnsmasq.service is the proxy DHCP + TFTP server.                                                                                                                                                     This service is handing out the PXE boot info and the iPXE (a modern open-source replacement for PXE) binary over TFTP (Trivial File Transfer Protocol).It does not hand out any IPs and keeps no leases.        The router hosts a full DHCP server.

 # Notes –

 this is the best we can do for right now

—

 # What is the service pxe-http.service

 The service pxe-http.service is a read-only HTTP server that serves files out of the directory /srv/http. This directory has everything iPXE calls for.                                                          The server itself is a simple Python program and lives at /etc/system/pex-http.service. At the moment the server only passes three things to the booting node.                                                   HEAD /arch/boot/x86_64/vmlinuz-linux, the Arch kernel, GET /slurm.conf the Slurm config, and GET /munge.key the Munge key. In addition to the HTTP server itself, it also has some supporting infrastructure.   The program itself waits till the network manager is online and it’s enabled by the system, so it will also start at boot.

 # Notes –

 This service is mostly ok, but I’m worried about security. mostly because I don’t understand what is going on, and I’m worried about a malfactor exploiting the service somehow if they

 were to get past the router layer or later, the security router which should be impossible, but idk hackers are scary.

 # //TODO –

 we have to rename the directory and to somthing that makes sence like /srv/http to /srv/pxe-server

———————————————

 # what is the service cluster-repo.service |

———————————————

 The service cluster-repo.service is a simple Python HTTP server that, as of right now hosts just SLURM and its dependencies.

 It’s similar to the service pxe-http.service in the sense that it’s just a local single package repo that the cluster can pull from. This service is also self-restarting.

 The directory it lives in is /srv/cluster-repo/ where all the files for this service live.  

 # //Notes –

 So this system is more or less completely fucked and does not do what we needed to do. Granted, I’m a moron and didn’t know what a mirror is or really what a repo is for that matter, but now I do.

 To summarize, this was supposed to be a local private curated mirror (so a repo, basically).

 The host has all the “cluster/system dependencies,” so shit like SLURM, the MUNGE keys, NetworkManager, fonts, whatever the fuck, etc.

 There are many reasons we want to do this, but the big one is the fact that in the year of the maker 2026, we have major governments and private parties attempting to destroy free and open-source internet

 computing and Arch Linux. So yeah, everything that we need to function needs to be self-hosted or else in like 3 years we are dead in the water.

 So to fix this, we need to tear this down and build a proper collection of curated mirrors.

 It would be a little dumb to put everything in one mirror, so we need a collection of mirrors so everything isn’t trying to butt fuck one port and to avoid re-downloading redundant shit like notofonts.

 # // TODO –

 Rebuild the mirror server. As for the architecture, to keep it organized, we should rename /svr/cluster-repo to srv/local-mirror-server.

 And from there he can have our separate directories for the separate mirrors. exsample /svr/local-mirror-server/SLURM-mirror, /NetworkManager, /hwclok, etc.

—————————————–

 # ACNST Tool Documentation             |

—————————————–

 ACNST is a custom C++ tool. It scans the cluster LAN using arp-scan and keeps a persistent device log keyed by MAC address. This is the tool that makes provisioning on a network with non-static IPs simple.

 The current cluster network is tied to my home network via DHCP, managed by the Xfinity router. The router hands out the IPs.

 That means when a fresh slave node PXE boots, we have no idea what IP it got, and nothing on the network remembers who is who between reboots, and the master node can’t talk to anyone.

 ACNST fixes this by keeping a living log of every device it has ever seen, keyed by MAC address, which stays the same between reboots.

 So even when the router shuffles IPs around, we always know “e0:be:03:… is arch001” and can see its current IP, online/offline status, and last-seen time.

 This program tries to keep a live map of the network using a systemd timer (acnst-scan.timer) that runs a scan every 30 minutes.

 # operation

 The operation is as simple as I could get it.

 # !we run all ACNST commands inside the ACNST directory!

 We can run the help command with – bin/acnst help,

 run a the scan command with – bin/acnst scan,

 We can vew the logs with the command – bin/acnst list,

 and we can set a hostname with the command bin/ acnst set-hostname –mac <MAC> –hostname <name>.

 It is important to tag nodes as soon as or before they come online.

 The ssh-list command prints the root@ for the online systems so scripts, robots, and humans don’t have to look up or hard code an IP.

 The ssh-list can pass that information to whatever needs it, we can call the ssh-list command with bin/acnst list.

 # Trubleshooting

 Troubleshooting: If a node is supposed to be running but the log shows it OFFLINE, something is wrong with that node or its cabling. Check the log first before digging deeper.

 If required, we have additional documentation such as full build, install, and test instructions in the ACNST repo’s README.

 (repo: https://github.com/REDACTED(coming soon)/, built with CMake; binary lives at bin/acnst)

 # dependencies – CMake v3.16, C++ compilerC++17 capable (GCC or Clang), arp-scan, glibc, make

——————————————————————————————————

 thinkcenter001 the master node and the first node|

 Networking

# Network management service :

 We are using the NetworkManager network manager. This is good because networking is managed by NetworkManager and auto-starts on boot.                                                                           systemd-networkd is also present and running, pulled in as a dependency/socket-activated, but is disabled and should not start on boot and reports enp3s0 as unmanaged—it manages no interfaces.                 And does not conflict with NetworkManager. We are using the default-named connection profile “Wired connection 1” (UUID 75c857cf-25b1-3302-8f43-8f84cd588bb3), set to a manual IPv4 config.                      There is no DHCP reservation and no custom dispatcher script involved. The static config lives entirely in this NM profile.

frame STL files

This section contains the STL files for ArchThinkCluster’s modular frame system, a 3D-printable rack structure built to house and organize compute nodes in a compact footprint. The frame breaks down into four printable components: a node cage for securing individual boards, stackable rack body sections (base and middle) that let the enclosure scale vertically as the cluster grows, and a rear wire cover for clean cable routing. Like the rest of this project, these files are a work in progress. Dimensions are currently sized for Lenovo ThinkCentre units only, and other platforms aren’t supported yet.