I helped write parallelknoppix when I was an undergrad - our university's 2nd cluster ended up being a bunch of laptops with broken displays running it. Took me a whole summer.

Then the next semester I am denied the ability to take a parallel computing class because it was for graduate students only and the prof. would not accept a waiver even though the class was being taught on the cluster me and a buddy built.

That I still had root on.

So I added a script that would renice the prof.'s jobs to be as slow as possible.

BOFH moment :)

I always wanted such thing for various "plumbing" services (DHCP/DNS/wifi controller etc) but lack of ECC and OOB management kinda disqualifies it for anything serious.

>He's running forty Blades in 2U. That's:

    >
    >      160 ARM cores
    >      320 GB of RAM
    >      (up to) 320 terabytes of flash storage
    >

>...in 2U of rackspace.

Yay that's like... almost as much as normal 1U server can do

Edit: I give up, HN formatting is idiotic

Ah, do you feel it too? That need to own some of these, even though you have zero actual use for them.

That is a neat setup. I wish someone would do this but just run RMII out to an edge connector on the back. Connect them to a jelly bean switch chip (8 port GbE are like $8 in qty) Signal integrity on, at most 4" of PCB trace should not be a problem. You could bring the network "port status" lines to the front if you're interested in seeing the blinky lights of network traffic.

The big win here would be that all of the network wiring is "built in" and compact. Blade replacement it trivial.

Have your fans blow up from the bottom and stagger "slots" on each row and if you do 32 slots per row, you probably build a kilocore cluster in a 6U box.

Ah the fun I would have with a lab with an nice budget.

These would be awesome for build servers, and testing.

I really like Graviton from AWS, and Apple Silicon is great, I really hope we move towards ARM64 more. ArchLinux has https://archlinuxarm.org , I would love to use these to build and test arm64 packages (without needing to use qemu hackery, awesome though that it is).

Multiple server vendors now have Ampere offerings. In 2U, you can have:

* 4 Ampere Altra Max processors (in 2 or 4 servers), so about 512 cores, and much faster than anything those Raspberry Pi have.

* lots of RAM, probably about 4TB ?

* ~92TB of flash storage (or more ?)

Edit : I didn't want to disparage the compute blade, it looks like a very fun project. It's not even the same use case as the server hardware (and probably the best solution if you need actual raspberry pis), the only common thread is the 2U and rack use.

This looks very promising. I basically could print an enclosure to specifically fit my home space. And easily print a new one when I move.

More efficient use of space compared to my current silent mini-home lab -- also about 2U worth of space, but stacked semi-vertically [1].

That's 4 servers each with AMD 5950x, 128GB ECC, 2TB NVMe, 2x8TB SSD (64c/512GB/72TB total).

[1] https://ibb.co/Jm1SX7d

The blade has arrived but can you get a compute unit to go in it? The non availability of the whole pi ecosystem has done a lot of damage.

I have this cycle every 10 years where my home infra gets to enterprise level complexity (virtualisation/redundancy/HA) until the maintenance is more work than the joy it brings. Then, after some outage that took me way too long to fix, I decide it is over and I reduce everything down to a single modem/router and WiFi AP. I feel the pull to buy this and create a glorious heap of complexity to run my doorbell on and be disapointed, can't wait.

I love the form factor. But please. For the love of god. We need something with wide availability that supports at least ARMv8.2.

At this rate I have so little hope in other vendors that we'll probably just have to wait for the RPi5.

If you want "hyperscale" in your homelab, the bare metal hypervisor needs to be x86-64 because unless you literally work for Amazon or a few others you are unlikely to be able to purchase other competitively priced and speedy arm based servers.

There is still near zero availability in mass market for CPUs you can stick into motherboards from one of the top ten taiwanese vendors of serious server class motherboards.

And don't even get me started on the lack of ability to actually buy raspberry pi of your desired configuration at a reasonable price and in stock to hit add to cart.

I’ve been getting good price/perf just doing the top AMD consumer CPU’s. Wish someone would make an AM5 platform motherboard with out of band / remote console mgmt. that really is a must if you have a bunch of boxes and have them somewhere else. The per core speeds are high on these. 16 core / 32 threads/boxe gets you enough for a fair bit.

I’ve built a small rasp k3s cluster with pi4 and ssd. It works fine but one can ultimately still feel that they are quite weak. Or put differently deploying something on k3s still ends up deploying on a single node in most cases and this gets single node performance under most circumstances

It's amazing to see how far these systems have come since my coverage from The Verge in 2014, where I built a multi-node Parallella cluster. The main problem I had then was that there was no of the shelf GPU friendly library to run on it, so I ended up working with the Gray Chapel project to get some distributed vectorization support. Of course, that's all changed now.

https://www.theverge.com/2014/6/4/5779468/twitter-engineer-b...

It's not clear to me how to build a business based on RPi availability. And the clones don't seem to be really in the game. Are Raspberry Pis becoming more readily available? I don't see that.

I really want something like NVidia's upcoming Grace CPU in blade format, but something where I can provision a chunk of SSD storage off a SAN via some sort of PCI-E backplane. Same form factor like the linked project.

I'm noticing that our JVM workloads execute _significantly_ faster on ARM. Just looking at the execution times on our lowly first-gen M1s Macbooks is significantly better than some of our best Intel or AMD hardware we have racked. I'm guessing it all has to do with Memory bandwidth.

Apple should go with a blade design for the Mac Pro. Just stick in as many M2 Ultra blades as you need to up the compute and memory.

Will need to deal with NUMA issues on the software side.

I have a few armada 8040 boards, and a couple raspberry pi's, but lets be real...

They're not going to get maximum performance from a nvme disk, the cpus are too slow, and gigabit isn't going to cut it for high throughput applications.

Until manufacturers start shipping boards with ~32 cores clocked faster than 2ghz and multiple 10gbit connections, they're nothing more than a fun nerd toy.

Been waiting for this for over a year, was the first person to buy a pre-purchase sample. Planning to set up a PXE k3s cluster.

This looks cool!

I would, however, say that while I'm in the general target audience, I won't do crowdfunded hardware. If it isn't actually being produced, I won't buy it. The road between prototype and production is a long one for hardware.

(Still waiting for a very cool bit of hardware, 3+ years later - suspecting that project is just *dead*)

> 160 ARM cores

> 320 GB of RAM

Depending how you feel about hyperthreading, there are commodity dual-CPU Xeon setups than can do this as well.

$60 per unit sounds pretty good. Does anyone have experience cross compiling to x86 from a cluster of Pis and can say how well it performs? A cheap and lower-power build farm sounds like an awesome thing to have in my house.

I want to build a fan out of them.

Probably with one of these in the middle: http://www.mercotac.com/html/830.html

This is a lot cheaper, more silent and smaller:

http://move.rupy.se/file/cluster_client.png

I think these are fantastic, but I really wish it had a BMC so one could do remote management. I'd love for version 2 to have it so I could buy a bunch for my datacenter.

There's no backplane - all power and communication goes through a front-facing ethernet port. Kind of defeats the purpose of a blade form factor IMO.

It's too bad ARM boards are so expensive, it makes them nearly pointless for projects unless you need the GPIO.

This is cool. But it's super hard to compete w/ a computer you bought off craigslist for 25$.

I'd like to buy a laptop that's also a fault tolerant cluster.

Love it, however, I'm skeptical of Raspberry Pi Foundation's claims that the CM4 supply will improve during 2023. It might improve for some, but as more novel solutions like these come up, the supply will never be enough.

How do we measure the performance of these kinds of systems?

These are pi's right? No hardware AES :/

Why only 1 Gbps ethernet?

Yes, this, more of this!

A

A

[dead]