[J-core] Microcontroller variant of J-Core?

[Rob Landley]

Yay, Wale fixed the admin interface so I can approve your posts that were stuck
in moderator approval! (The problem was we added https to j-core.org, with an
automatic http->https redirect, and thus hit
https://mail.python.org/pipermail/mailman-users/2016-October/081504.html because
a regex hadn't got updated.)

Sorry for the delay replying, I had to chase a tangent. :)

On 5/18/20 1:04 AM, Andrew Pullin wrote:
> Wow, all very interesting and informative. Some thoughts follow.
> Broadly: I looked at the ice40 repo and and it still seems a little impenetrable
> to me given my novice skill level.

I'm happy to walk you through it. Jeff's walked _me_ through it at some length.
(I even blogged about it, I think?)

> But I still see big potential in making a free-and-in-the-clear core an
> off-the-shelf component.

See the post I just made to the list about the j2 repo going up on github.

  https://lists.j-core.org/pipermail/j-core/2020-May/000910.html

> On 5/17/2020 4:50 PM, Rob Landley wrote:
>> We're using fpga sram for lots of stuff. In fact all the memory the ice40
>> implementation has is 2 different kinds of sram blocks, the j2 register file on
>> xilinx is sram, the L1 cache is sram...
>>
>> If you mean bolting on _external_ sram, I think the limiting factor is that's
>> not how we wanted to spend the available iopins in any of the FPGAs we're using.
> 
> Yeah, I really meant: make a functional MCU core with existing resources
> on-chip, in a configuration that mimics what you'd find in a mid-range MCU.

It works for us and we've shipped working systems to customers? (See the
spacechain press release linked from the above post.)

> Although MCU's with SDRAM/PSRAM certainly exist out there for graphics
> applications. And all the midsize FPGA eval boards I have include an SDRAM (e.g.
> Digilent Arty) or DDR3L (e.g. OrangeCrab, ECP5).
> That's part of the configuration space that I was thinking might be possible to
> span, from a resource equivalent of an M0 (on-chip) to an M7 (off-chip, and
> enough IO to support the RAM).

What we did is write an lpddr2 controller in VHDL, which lets us glue up to 256
megabytes of DRAM to the system, and that's the context it runs Linux in. (In
theory we could add a switchy thing to the memory bus and use multiple lpddr2
chips and get up to 4 gigabytes, but that gets big and expensive and power
consuming fast; a better approach would be to update the memory controller to
ddr3 or something when we want to go there. But so far, that hasn't been a
significant limiting factor in the systems we've been putting together...)

This hardware was present on the old Numato Mimasv2 boards, and it's also
present on our Turtle boards. (Which exist, I have one here, and we REALLY need
to get those up on crowd supply.)

There was a LOVELY talk at the 2015 embedded Linux Conference about running
Linux in 256k of SRAM using nor flash and configuring everything to XIP out of
that like it was ROM (https://elinux.org/Kernel_XIP and
https://elinux.org/Application_XIP). The fact it was nommu was important to this
because it meant you didn't need to waste memory on page tables. :)

Alas, the Linux Foundation accidentally deleted a couple years of talks off of
youtube recently, and that was one of the recordings that went. But the PDF of
his slides is still available:
https://elinux.org/images/9/90/Linux_for_Microcontrollers-_From_Marginal_to_Mainstream.pdf

What we did instead on the j-core ice40 build is use 4 of the SRAM blocks as
caches with a little hardware blob managing them doing spi flash transactions to
read data into that cache line every time there's an address lookup miss in the
relevant address range. But that's for running bare metal code, not Linux. (And
that's not in the stuff we uploaded to github so far, we're still debugging it.)

When most people say "embedded" (a term that came from military hardware) they
usually just mean "usually-headless box running a dedicated app instead of being
a general purpose computer", and _those_ guys will tell you that if you really
REALLY squeeze you can get Linux down into 2 megs of ram with things like Matt's
old -tiny tree:

  https://en.wikipedia.org/wiki/TinyLinux

(Which again collected patches that went upstream so the out of tree fork wasn't
needed anymore. But looking at the out of tree fork lets you know what to switch
on in the configuration. :)

>>> The curiosity comes from what I perceive as a trend in the FPGA/IC sector at the
>>> moment:
>>> Seeing that ARM is making their M0/M1 cores available for free trial suggests
>>> that they feel the pressure of RISC-V.
>>
>> Nah, it's because the superh patents expired a few years back, which Arm had
>> licensed to create the thumb instruction set, meaning chips implementing the
>> thumb instruction set are now significantly cheaper, which put the thumb-only
>> cortex-m* into a new price category that ARM is trying to take advantage of by
>> pumping up the volume. Cortex can now be a "low margin, high volume"
>> microcontroller without having to pay third party royalties on every chip sold.
>> (ARM still gets paid, but Hitachi/Renesas doesn't get tens of millions annually
>> in patent license fees on top of that.) And the BIG news there is that FDPIC for
>> Arm finally got merged into gcc and Linux, which makes Linux cortex-m a lot
>> cleaner.
> 
> Linux Cortex-M? That's ... interesting.

It's been around for years. The out of tree patches were circulating in 2010,
and a moscow company picked them up and made a business out of them:

  https://emcraft.com/
  https://github.com/EmcraftSystems/linux-emcraft/commit/a2bb3d68e397

Which you get to if you scroll back about 500 commits in:

  https://github.com/EmcraftSystems/linux-emcraft/commits/linux-cortexm-1.12.1

And some french dudes did FDPIC support for arm a few years back:


https://www.slideshare.net/linaroorg/sfo15406-arm-fdpic-toolset-kernel-libraries-for-cortexm-cortexr-mmuless-cores

Then last year the arm fdpic support finally went upstream, the hard part of
which was (as always) the FSF's copyright assignment paperwork nonsense:

  https://gcc.gnu.org/legacy-ml/gcc-patches/2019-09/msg00533.html

(Honestly, just add all the missing architecture support to llvm and let the FSF
die already.)

> Having just gone through a job search, I have seen that "embedded" encompasses
> both the Linux and the microcontroller world, and few (if any) people have both
> in their skill stack.

If any? Jeff's been COLLECTING people who know this stuff, and if you go back
far enough he's the one who _taught_ Linux most of this (back in uclinux).

Back when I used to maintain busybox I tried to document some of the nommu
programming pitfalls:

https://git.busybox.net/busybox/tree/docs/busybox.net/FAQ.html?id=b73d2bf4bfac#n716

I was just getting into it, but circa 2006 we had a guy gluing busybox to
newlib+libgloss to run it on bare metal hardware, and I've used several u-boot
variants over the years that run busybox "hush" as its shell and call busybox
apps as builtins from it.

I originally got into busybox because I was trying to create the simplest Linux
system capable of building itself under itself from source (and building Linux
>From Scratch under the result as a proof you could natively bootstrap up to an
arbitrarily complex system, up to and including building existing distros like
red hat or debian). I got this minimal native development environment down to 7
packages (busybox, uclibc, gcc, binutils,make, bash, linux):

  https://landley.net/aboriginal/about.html

My inspiration at the time was the tomsrtbt rescue floppy providing a complete
Linux system in 1.7 megabytes, and I wanted to fit a complete self-hosting
development environment in something similar (using the 100k tinycc executable).
That's why I maintained a tinycc fork for 3 years (https://landley.net/hg/tinycc).

I then conceptually simplified this "simplest native development environment"
down to 4 packages (kernel, libc, compiler, cmdline) which would currently be
linux, musl, qcc, and toybox. I replaced the build system with a similarly
simplified 250 line shell script:

  https://github.com/landley/toybox/blob/master/scripts/mkroot.sh

Which is now a part of toybox, the project I started when I left busybox, which
was merged into Android starting with Marshmallow in 2014 and has provided the
android command line utilities ever since:

  https://landley.net/toybox/about.html

I wrote up all this history long ago, and gave a talk about it last year:

  http://landley.net/aboriginal/history.html
  https://www.youtube.com/watch?v=MkJkyMuBm3g
  http://landley.net/talks/elc-2019.txt

Something I'd like to do NOW is take MIT's xv6 project, port it to j-core, and
write a corresponding hardware-side textbook about the VHDL.

  https://pdos.csail.mit.edu/6.828/2014/xv6.html

(I refer to this as "coresemi's educational arm". We could sell jduino kits with
our ice40 platform... you can do all sorts of cool stuff with jcore if we can
just get the people and time pointed at it...)

Admittedly xv6 has the minix/pascal problem of "aims to be a teaching tool, not
real world useful, and thus has some very obvious holes if you try to do
anything with it". (It doesn't even have mmap.) So every time I mention it Jeff
goes "can't we use a real RTOS instead?" and he has a point. But the interesting
part about xv6 is the _textbook_, not the codebase. It was created by the MIT
guys taking the classic Lions' book
(https://en.wikipedia.org/wiki/Lions%27_Commentary_on_UNIX_6th_Edition,_with_Source_Code)
and modernizing it (from K&R code on a PDP-11 to c99 code that runs on
qemu-system-386).

People have made various posix extensions of xv6, the same way there was a 386
port of minix. And they can never go upstream for the same reason: teaching tool. :(

> I wasn't aware there was a push for a hybrid or a middle-ground solution like
> that...

My boss is Jeff Dionne, who long ago was the founder of uclinux and currently
the CEO of CoreSemi, I.E. he runs the company that does most of the j-core
development.

He's the guy who ported Linux to nommu hardware back in the 1990's. Jeff moved
to Japan in 2003 and handed uclinux off to people who didn't maintain it (as in
years alter all their repos were still in a single un-backed-up CVS server when
its hard drive died), but it had already served its purpose (the hardware
support went upstream into the various packages and you can use buildroot or a
hacked up debian or your own LFS variant on nommu systems now if you want to).

Jeff sort of emerged from the hardware world into Linux development and then
went back to focusing on hardware, but he's straddled both forever. He was Erik
Andersen's boss at Lineo (he's the one who hired him to maintain busybox and
uclibc).

We're NOT UNIQUE by the way. Yes we pulled Rich Felker (maintainer of musl-libc)
into the nommu world a few years back, but guys like Greg Ungerer and Geert
Uytterhoeven work completely independently from us, although we know about them
because we interact online semi-regularly:

  https://www.spinics.net/lists/linux-sh/msg56959.html
  https://www.spinics.net/lists/linux-sh/msg56987.html

Yoshinori Sato introduced Linux support for the sh2 architecture and maintains
the H8/300 architecture (both nommu), and he was doing that years before we
tracked him down and bought him lunch (he lives in Tokyo) and asked him if he'd
co-maintain superh with Rich when we were bringing that back into proper support.

Like Jeff, these guys all know more about this stuff than I do (and have been
doing it longer). :)

>>> And there is always Microblaze in certain toolchains, but that is a licensed IP
>>> product married to a specific ecosystem.
>>
>> Microblaze isn't a product, it's a firmware package designed to sell FPGAs.
>> Xilinx never wanted to make an ASIC out of it, that's not what it's for. But
>> half the people who make xilinx FPGA boards bolt either an atmel or a cortex-m
>> onto the side of the FPGA to load the control it (otherwise reflashing is really
>> bricky), and unless the microblaze is significantly more capable than that ASIC
>> why bother except as a learning exercise?
> 
> Yes, now you're really driving into the core of my curiosity:
> As I am sure you know, the division between CortexM+RTOS and "Linux with DDR"
> really divides the embedded device world.

Define "divides".

  https://www.youtube.com/watch?v=8Gv0H-vPoDc

Jeff pointed me at an 8 megabyte sram chip back in February, although it was
only avilable from one vendor and a bit expensive for what we were trying to do
(and there was an I/O data rate bottleneck because of the ice40 pin shortage),
but the point is we probably CAN run Linux on the ice40 j-core if we really try. :)

But right now we're doing bare metal stuff with that...

> In the robotics/hobby space, the combination of CortexA+Linux e.g. Raspi and a
> bare-metal MCU for sensor/motor control is entirely standard, but still a pretty
> messy landscape of technologies.

"Messy landscape of technologies" is normal anywhere a technology hasn't matured
yet. Simple is a result that takes WORK, and "Making it look easy" means you've
mastered it.

For over a year my email .sig was this quote:

  https://www.brainyquote.com/quotes/antoine_de_saintexupery_103610

This Ken Thompson quote was a margin note he scribbled in red pen on a printout
of the first 8 chapters of The Art of Unix Programming (he "reviewed" the book
by having a draft printout mailed to him, which he physically wrote his comments
on and mailed back).

  https://www.brainyquote.com/quotes/ken_thompson_254858

See also Pascal's Apology:


https://www.npr.org/sections/13.7/2014/02/03/270680304/this-could-have-been-shorter

There's a REASON I'm proud to have written a 232 line sntp.c or a 502 line
grep.c in toybox. My 483 line patch.c now includes has fuzz factor support.
Writing a BIGGER one would have been easy, the work is to make it SIMPLE.

It's similar with j-core: Jeff's created a processor written using the European
Space Agency's VHDL "two process method" that can run Linux and fits in an
ice40. That took a lot of polishing, and we're still polishing it. (The
traditions of dorodango and kintsugi are _also_ japanese.)

> I am not targeting ASIC production, more like trying to take a step back from
> those standard solutions and see what might be possible now, given that
> previously exotic technologies are in reach in terms of complexity and cost
> constraints. Zynq+softcore as the next step in Raspi+STM32, something like that.

See the "practical hardware: FPGAs vs ASICs" slide from our 2016 j-core talk:

  https://j-core.org/talks/ELC-2016.pdf

That video was ALSO one of the ones destroyed by the Linux Foundation in The
Great Oops of 2019:

  https://www.spinics.net/lists/linux-embedded/msg04243.html

But I have a copy on my website:

  https://landley.net/talks/jcore-2015.mp4

> To be lofty about it: What Arduino did for microcontrollers in the past 10-15
> years, I see FPGA tech approaching a similar inflection point in accessibility
> and use.

Jeff saw it doing that in about 2012. I think it was in one of our talks (we
gave like 3 at different conferences, that's just one of them. Don't remember
which material was in which talk.)

>> RISC-V always struck me as open source's itanium. It's a solution in search of a
>> problem that tries to make a self-fulfilling prophecy out of marketing itself as
>> "inevitable" the same way Hillary Clinton did in 2008 _and_ 2016. But I'm the
>> first to admit I'm biased here. :)
> 
> Well, RISC-V is on my radar because it is pushing into the microcontroller
> embedded space a bit.

Oh it tries very hard to be all things to all people. Every time Jeff pointed
out a case they'd got wrong they'd add another instruction set extension to win
that particular microbenchmark. (He eventually learned to stop doing that.)

To me, the main difference is that j-core was made by engineers trying to solve
problems and coming up with tools they actually used along the way, dogfooding
it ("eating their own dog food") from day 1. Meanwhile RISC-V started with some
ivory tower academics spending years on an intellectual exercise, and eventually
trying to find some real world application for it after the fact.

I've watched J-core was develop like C and Unix and Linux did, and RISC-V
develop the way Pascal and Minix did. The name would suggest RISC-V is trying to
be more like AT&T's System V, but Unix took over the world because because the
guy who created the bell labs "dogfooded" version took a year off to teach at
his alma mater, and those grad students dogfooded their fork to produce BSD
which got the Darpa Vax contract in 1979 to replace the honeywell IMPs so
starting in 1980 everything plugged into the internet was using BSD Unix to do
it. Then after the 1983 Apple vs Franklin lawsuit extended copyright to cover
binaries and triggered the shrinkwrap software gold rush, AT&T allowed itself to
be broken up to get out from under the 1957 sherman antitrust consent decree
that kept it from selling software and went around paying all the unix vendors
to replace BSD with System V (resulting in flag day rewrites like SunOS ->
Solaris), and the result was "the unix wars" that almost killed the platform
until Linux came along and rendered AT&T moot.

Did I mention computer history's a hobby of mine?

  https://landley.net/history/mirror

> Vendors are supporting it very early, since they see signs
> that it might supplant ARM due to being "free" (I think). Chinese industry has
> clearly invested heavily in it for their Application Processor class of devices.

Remember when a MIPS variant was the official national processor of China?

https://venturebeat.com/2015/09/03/chinas-loongson-makes-a-64-bit-mips-processor-that-can-run-x86-and-arm-code/

I suspect that was somehow fallout from the whole Lexra debacle, but never got
the details from china's side:

  https://landley.net/notes-2009.html#14-12-2009

But sure, they're doing it. Not saying they aren't. It's just of zero interest
to _me_.

> When IAR and Segger are both supporting it before IC's are even available,
> that's notable.
> 
> That's also some of the source of my nebulous idea and interest in the
> reconfigurable computing space:
> The promise of RISC-V was that you can put a little element of firmware-driven
> control into your system anywhere you want, so maybe replacing a high-clock MCU
> with an ice40 and specific peripherals might be better, faster to put together,
> lower NRE's, etc.

You're aware that was in the microblaze marketing material over a decade ago,
right? And of course:

  https://www.kernel.org/doc/html/v4.18/driver-api/fpga/fpga-mgr.html

Was introduced in the v4.4 kernel in 2015...

But sure, yeah, the fact this was all being done before doesn't preclude them
from taking credit for it. They do about 5 times as much marketing as
development, as far as I can tell...

> For example: If the SiFive folks bundled an RV32i core as a piece of "Vivado IP"
> that could be dropped in a block design for a Zynq or other 7-series, connected
> to AXI and a QSPI I/O, and debugged easily, I think some heads could be turned.

Um, yes? (You're aware opencores.org exists and has for some time, right? But
"we can do that too" is an interesting rallying cry, sure.)

I mean hey, maybe they'll succeed. Java popularized garbage collection, a
technique John McCarthy invented in 1959 but everybody thought was new in Java.

> That is a personal projected I wanted to try and accomplish, but, again, not
> sufficiently skilled yet. Looking at J-Core was along the same lines.

Jeff's the guy you want to talk to about that.

>>> Do the authors & experts here think that J-Core might be a candidate for that
>>> application?
>> Sure.
>
> OK, so:
> If someone brought together a single repo with J-Core + toolchain + tooling +
> makefile to effectively get someone to a C or Rust main() and memory-mapped RTL
> modules out-of-the-box, I think this could be part of that aforementioned
> FPGA-accessibility inflection point.

Another thing they could do is create a $100 price point board in a raspberry pi
2b form factor with an lx45 and all the raspberry pi I/O devices, lpddr2 memory,
and get Linux booting on it.

We wanted to have a bitstream driving all those I/O devices before we put it up
on crowdsupply, and we STILL haven't quite had cycles to finish that:

  https://lists.j-core.org/pipermail/j-core/2020-January/000900.html

The perfect is the enemy of the good and we were juggling slightly more balls
than is advisable BEFORE coronavirus. I suspect we should just ship it and push
updates to github as we get 'em, but "test all the I/O ports by hand in our
prototype run" and "make sure we're shipping out a good board to each customer"
are not the same thing. There's todo items to get stuff into actual
_production_, even at low volumes...

> (I did not realize yosys did not support VHDL, so I don't know how realistic
> that is yet)

We can build it and run it for ice40 with a fully open source toolchain now:

  https://lists.j-core.org/pipermail/j-core/2019-November/000868.html

(Although Jeff says they had a regression over the weekend and it no longer
builds a working ice40 bitstream, and he hasn't had time to debug it yet. But
roll back your checkout a few days and it should work fine?)

The xilinx side hasn't got a working open source place and route yet, that's the
main reason we're still using the proprietary toolchain there.

> But that sounds like another forever-project.
> If it's something you're working on, though, I'll watch the repos for it to appear.

The thing is, we've been doing a lot of forever-projects for a number of years
now, and if you grind away at it long enough you eventually get stuff to work. :)

I believe j-core development started in something like 2012? I came onboard
towards the end of 2014. We got SMP Linux support merged into linux-kernel in:

  commit c8e67a850b7ee314c54fc47d8faa4d331dbeafd9
  Author: Rich Felker <dalias at libc.org>
  Date:   Wed Apr 27 00:11:38 2016 +0000

      sh: add defconfig for J-Core J2

But we spent a LOT of cycles along the way working on customer devices and whole
other subsystems like creating a new GPS implementation from scratch. Which also
just went up on github:

  https://news.ycombinator.com/item?id=23234771

Workin' on it...

> - Andrew

Rob