Alpha/Linux Wiki

The original Alpha architecture has no byte and no 16-bit word memory instructions: every integer load and store is a longword or a quadword, and byte and word access is synthesized from LDQ_U/STQ_U plus the extract, insert and mask instructions. The byte/word extension (BWX), introduced with the EV56 (21164A), added byte and word loads and stores.

Linux 6.10 removed support for processors older than the EV56, and Debian builds for EV56 or later, so every current Alpha/Linux system has BWX. ¹⁾ For software built today the rules are short:

• Build for EV56 or later. With -mcpu=ev56 (or pca56, ev6, ev67), byte and word loads and stores are single instructions, and a byte or word store does not disturb its neighbors. The compiler's default may still be the pre-BWX baseline (see Distribution baselines); __alpha_bwx__ is defined when BWX code generation is enabled, so gcc -dM -E - < /dev/null | grep __alpha_bwx__ shows whether it is.
• Assemble for EV56 or later. Without .arch ev56 or -mcpu=ev56, the assembler silently expands stb and stw into non-atomic pre-BWX sequences (see Toolchains).
• Byte and word atomics are still emulated. There are no byte or word load-locked or store-conditional instructions, even with BWX; atomic operations on char and short objects operate on the containing quadword (see below).
• Keep independently updated atomic variables apart. A store-conditional fails if another processor writes anywhere in the locked range, which is at least a quadword and often more (see Reservation granularity).

The pre-BWX sequences and the sparse I/O space that the original design required are described in the later sections of this page, for reading old code and kernel sources.

See also Unaligned Access, Memory Model and Architecture Mask (amask).

BWX comprises six instructions, LDBU, LDWU, SEXTB, SEXTW, STB, and STW, and is reported by bit 0 of AMASK. ²⁾

There is no sign-extending byte or word load; the equivalent is LDBU followed by SEXTB.

The Architecture Reference Manual states which form to prefer:

LDBU and STB are the recommended way to perform byte load and store operations on Alpha implementations that support them; use them rather than the extract, insert, and mask byte instructions described in this section.

Linux does not emulate these instructions: a process that executes one on a pre-EV56 processor receives SIGILL.

alpha-unknown-linux-gnu-gcc -mcpu=ev56 …    # or pca56, ev6, ev67
alpha-unknown-linux-gnu-gcc -mbwx …         # explicit
alpha-unknown-linux-gnu-gcc -mno-bwx …      # force the pre-BWX sequences

With BWX enabled, ordinary C produces single instructions:

lb:	ldbu $0,0($16)
lsb:	ldbu $0,0($16) ; sextb $0,$0
sb:	stb  $17,0($16)
lw:	ldwu $0,0($16)
sw:	stw  $17,0($16)

Two further flags, added in GCC 15, address the non-atomicity described below: ³⁾

The compile-time predicate is __alpha_bwx__, defined when BWX code generation is enabled.

GCC has no builtins for the BWX instructions: __builtin_alpha_ldbu, __builtin_alpha_ldwu, __builtin_alpha_stb and __builtin_alpha_stw do not exist. The instructions are obtained by compiling with -mbwx and writing ordinary unsigned char and unsigned short accesses. GCC provides builtins for the pre-BWX byte-manipulation instructions: __builtin_alpha_extbl, __builtin_alpha_insbl, __builtin_alpha_mskbl and the rest of the extract, insert and mask family, along with __builtin_alpha_amask, __builtin_alpha_implver, __builtin_alpha_cmpbge, __builtin_alpha_zap and __builtin_alpha_zapnot.

amask(1) == 0 indicates that BWX is present. See Architecture Mask (amask) for the instruction, its bit assignments and the kernel's AT_HWCAP export.

On a pre-BWX Alpha, storing a byte means loading the containing quadword, modifying one byte of it, and storing the whole quadword back. If another processor stores to a different byte of the same quadword in between, that store is silently lost.

C and C++ both require that distinct scalar objects can be updated concurrently without interfering. Pre-BWX Alpha cannot honor that for char and short without -msafe-bwa, which GCC 15 added for this purpose. ⁴⁾ The Linux kernel likewise assumes atomic byte stores.

Michael Cree, the Debian Alpha port maintainer, estimated that possibly a third of the bugs in the port arose from non-atomic byte and word accesses, and noted that kernel developers wanted to assume atomic byte and word access on all architectures, which only pre-BWX Alpha prevented. ^{5) 6)}

Debian Ports raised its Alpha baseline to EV56 in 2023, and Linux followed in 6.10. Alpha EV5 and earlier was the only CPU family the kernel supported without native byte access, and parts of the RCU code already relied on byte access and did not work reliably on EV5. ⁷⁾ Linux 6.10 dropped pre-EV56 support, and removed Jensen, LCA and APECS support along with it.

Linux and Debian therefore require EV56 or later.

The locked primitives exist only at longword and quadword granularity:

• LDL_L / STL_C, 32-bit
• LDQ_L / STQ_C, 64-bit

There is no LDB_L, STB_C, LDW_L or STW_C, and BWX adds none. Atomic byte and word operations must be synthesized, on every Alpha, from the quadword primitives plus masking.

GCC generates the following for __sync_fetch_and_add((unsigned char *)p, 1) at -mcpu=ev4:

afa:
	bic $16,7,$3          /* round down to a QUADWORD */
	ldq_u $1,0($16)
	extbl $1,$16,$1
$L8:
	mb
	addl $1,1,$2
	and $1,0xff,$5
	insbl $2,$16,$2
$L11:
	ldq_l $6,0($3)        /* load-locked the containing quadword */
	extbl $6,$16,$1
	cmpeq $1,$5,$4
	beq $4,$L12
	mskbl $6,$16,$4
	bis $4,$2,$4
	stq_c $4,0($3)
	beq $4,$L11
	mb
$L12:
	beq $4,$L8
	mov $5,$0
	ret $31,($26),1

Three points follow:

• The alignment mask is bic $16,7, rounding to a quadword rather than a longword, because extbl, insbl and mskbl index by the low three bits of the address.
• The primitive is LDQ_L/STQ_C. A hand-written version using LDL_L with quadword-relative extract and insert offsets silently corrupts memory for every address whose bit 2 is set.
• At -mcpu=ev56 the loop body is identical; only the initial load becomes ldbu. BWX changes nothing here.

The reservation is neither per-byte nor per-longword. From the Architecture Reference Manual:

When a LDx_L instruction is executed without faulting, the processor records the target physical address in a per-processor locked_physical_address register and sets the per-processor lock_flag.

If processor A's lock_flag is set and processor B successfully does a store within A's locked range of physical addresses, then A's lock_flag is cleared. A processor's locked range is the aligned block of 2**N bytes that includes the locked_physical_address. The 2**N value is implementation dependent. It is at least 8 (minimum lock range is an aligned quadword) and is at most the page size for that implementation (maximum lock range is one physical page).

and, on what to do about it:

• Hardware implementations are encouraged to lock no more than 128 bytes.
• Software implementations are encouraged to separate locked locations by at least 128 bytes from other locations that could potentially be written by another processor while the first location is locked.

The granularity relevant to contention is therefore an implementation-defined aligned block, which may be larger than a cache block. Two atomic variables in the same lock range cause each other's STQ_C to fail, even if they are in different cache blocks; separating them by 128 bytes, as the manual recommends, avoids this on every implementation.

/* BAD: two independently-updated atomic bytes inside one lock range.
   Every update to one makes the other's STQ_C fail and retry. */
struct {
	_Atomic unsigned char producer_flag;
	_Atomic unsigned char consumer_flag;
} state;
 
/* BETTER: separate them by the architecture's recommended distance. */
struct {
	_Atomic unsigned char producer_flag;
	char pad[127];
	_Atomic unsigned char consumer_flag;
} state_padded;

Folding the flags into a single word and using bit operations avoids the problem entirely, leaving one contended location:

_Atomic uint32_t flags;
 
atomic_fetch_or_explicit(&flags,  1U << 0, memory_order_relaxed);
atomic_fetch_and_explicit(&flags, ~(1U << 1), memory_order_relaxed);

Sustained contention can starve an individual thread, but cannot stall the system. The architecture requires that:

If two processors attempt STx_C instructions to the same lock range and that lock range was accessed by both processors' preceding LDx_L instructions, exactly one of the stores succeeds.

The portability hazard lies elsewhere, and is covered under Forward progress rules: an LDx_L/STx_C body containing any other memory access, a taken branch, a subsetted instruction, or too many instructions may fail every time on some implementations.

This section describes the code generated for processors without BWX, which current kernels no longer support.

	ldq_u $0,0($16)      /* aligned quadword containing the byte */
	extbl $0,$16,$0      /* extract, selecting the byte by $16<2:0> */

EXTBL uses the low three bits of its second operand, that is, the byte offset within a quadword. This is significant for the atomic sequences described below.

A signed byte costs two more instructions, because there is no sign-extending load:

	ldq_u $0,0($16)
	extbl $0,$16,$0
	sll $0,56,$0
	sra $0,56,$0

A store is a read-modify-write of the surrounding quadword. The Architecture Reference Manual gives the canonical sequence:

	LDA      R6, X(R11)    ; R6<2:0> = (X mod 8) = 5
	LDQ_U    R1, X(R11)    ; Ignores va<2:0>, R1 = yyAx xxxx
	INSBL    R5, R6, R3    ; R3 = 00A0 0000
	MSKBL    R1, R6, R1    ; R1 = yy0x xxxx
	OR       R1, R3, R1    ; R1 = yyAx xxxx
	STQ_U    R1, X(R11)

The sequence is five instructions plus the address materialization, where x86 uses a single MOV. GCC emits the same form, using insql rather than insbl (equivalent when the source register already holds a zero-extended byte):

	insql $17,$16,$17
	ldq_u $1,0($16)
	mskbl $1,$16,$1
	bis $17,$1,$17
	stq_u $17,0($16)

This read-modify-write is not atomic, which is the source of a large class of defects on Alpha. Two processors storing to two different bytes of the same quadword can lose one of the two stores entirely. See Atomicity of Byte and Word Access.

An aligned word is two instructions, exactly like a byte:

	ldq_u $0,0($16)
	extwl $0,$16,$0

An unaligned word may straddle a quadword boundary, so it needs both halves:

	ldq_u $0,0($16)
	ldq_u $1,1($16)
	extwl $0,$16,$0
	extwh $1,$16,$1
	bis $0,$1,$0
	zapnot $0,3,$0

The unaligned case is the longest of these sequences: two read-modify-writes, ten instructions.

	LDA      R6, X(R11)    ; R6<2:0> = (X mod 8) = 5
	LDQ_U    R2, X+1(R11)
	LDQ_U    R1, X(R11)
	INSWH    R5, R6, R4
	INSWL    R5, R6, R3
	MSKWH    R2, R6, R2
	MSKWL    R1, R6, R1
	OR       R2, R4, R2
	OR       R1, R3, R1
	STQ_U    R2, X+1(R11)  ; Must store high then low for
	STQ_U    R1, X(R11)    ; degenerate case of aligned

The Architecture Reference Manual's comment states the ordering constraint: the high quadword must be stored before the low one, so that the degenerate case where both LDQ_U hit the same quadword still produces the right answer.

An aligned word store is a single read-modify-write, as for a byte.

An EV4 cannot issue a byte or word write to the bus, while ISA and EISA devices require byte-granular access. The transfer length and the byte lane are therefore encoded in the physical address, and the chipset decodes them from a longword access.

For the mainstream chipsets (APECS, LCA, CIA/Pyxis, T2) the layout is:

Physical address = IO_BASE + (io_port << 5) + ((length - 1) << 3)

In practice the offsets are 0x00 for a byte, 0x08 for a word and 0x18 for a longword. The encoded value for a longword is 3, not 2; value 2 denotes a tribyte transfer.

The chipset performs no byte shifting. From the DECchip 21171-CA Technical Reference Manual:

Software must use longword load or store instructions (LDL/STL) to perform a reference that is of longword length or less on the PCI bus. The bytes to be transferred must be positioned within the longword in the correct byte lanes as indicated by the PCI byte enable. The hardware will do no byte shifting within the longword.

Software must place the byte in the correct lane. Linux uses __kernel_insbl(b, addr & 3) on write and __kernel_extbl(result, addr & 3) on read:

__EXTERN_INLINE void cia_iowrite8(u8 b, void __iomem *xaddr)
{
	...
	w = __kernel_insbl(b, addr & 3);
	*(vuip) ((addr << 5) + base_and_type) = w;
}

Only the Jensen replicated the byte to all four lanes.

Barriers are mandatory rather than advisory. From the same manual:

Programmers are required to insert memory barrier (MB) instructions between sparse space accesses to prevent collapsing in the 21164 write buffer.

Sparse access is markedly slower than dense: every read is a load plus an extract, every write a load plus an insert, each needs an MB, and consecutive ports are 32 bytes apart so nothing coalesces.

A dense mapping is linear: physical byte N corresponds to bus address N, with no stride and no lane encoding.

Dense space is a chipset feature rather than a CPU feature. Pre-BWX chipsets such as APECS, LCA and T2 do provide a dense memory window, usable for longword and quadword access on an EV4. They cannot provide byte and word access through it, so Linux converted a dense address back into the sparse window to service readb/writeb. From the APECS support removed in Linux 6.10:

	if (addr >= APECS_DENSE_MEM) {
		addr -= APECS_DENSE_MEM;
		APECS_SET_HAE;
		base_and_type = APECS_SPARSE_MEM + 0x00;
	}

Byte/word-capable windows arrived with Pyxis, which added them as separate windows rather than making the existing dense window byte-capable:

#define CIA_DENSE_MEM		(IDENT_ADDR + 0x8600000000UL)
#define CIA_BW_MEM		(IDENT_ADDR + 0x8800000000UL)
#define CIA_BW_IO		(IDENT_ADDR + 0x8900000000UL)
#define CIA_BW_CFG_0		(IDENT_ADDR + 0x8a00000000UL)
#define CIA_BW_CFG_1		(IDENT_ADDR + 0x8b00000000UL)

Tsunami, Polaris and Irongate likewise. The kernel expresses the capability per machine vector, not by a runtime CPU check:

#define t2_trivial_rw_bw	0        /* T2: must use sparse */
#define cia_bwx_trivial_rw_bw	1        /* Pyxis: plain dereference */
#define tsunami_trivial_rw_bw	1
#define irongate_trivial_rw_bw	1

trivial_rw_bw = 1 means readb/writeb are ordinary dereferences, which the compiler turns into LDBU/STB when built for BWX. Byte and word I/O therefore requires both a BWX CPU and a byte/word-capable chipset; neither alone is sufficient.

Linus Torvalds stated the rule that has been kernel policy since, on the Alpha list in 1996: ⁸⁾

You MUST NOT access IO memory any other way than read/write[bwl](). or memcpy_fromio/toio(). EVER. […] On other architectures you'll only see a _window_ into the IO space, so that the IO access functions have to set up the window correctly, and the same processor virtual address can actually be _different_ IO ports depending on where the window happens to be. Or look at the Jensen, for example: it has _no_ support for dense memory, and you _cannot_ make the IO memory look like a memory access.

That "window" is the HAE (host address extension) register, which supplies the high address bits that do not fit in the sparse encoding. Every sparse access has to set it, and the kernel caches its value in alpha_mv.hae_cache to avoid rewriting it unnecessarily.

A processor with BWX can make byte-granular accesses to I/O space only if the core logic also implements them. Byte and word access to I/O space was reported never to have worked on the 21172 (CIA-2), with Pyxis the first core logic to implement it correctly, although FreeBSD later enabled it on a 21172-based AlphaServer 800 without reported problems. ⁹⁾

The Jensen (DEC 2000 AXP, DECpc AXP 150), an early EV4-based EISA machine whose Linux support was removed in 6.10, predates these conventions: it used a 128-byte sparse stride, replicated bytes to all four lanes, had no dense space, and reached its keyboard, RTC, parallel and serial ports through a separate local I/O window.

• Alpha Architecture Reference Manual, Fourth Edition. §4.6 covers the extract, insert and mask instructions and the canonical byte and word sequences; §4.11.1 covers AMASK; Appendix D, Table D-3 gives the AMASK bit assignments. The 1992 and 1995 editions predate BWX and AMASK; the Alpha Architecture Handbook, Version 4 also documents both.
• Alpha Architecture Reference Manual, First Edition, §4.2, for the LDx_L/STx_C lock range definition.
• DECchip 21171-CA Technical Reference Manual, Chapter 6, for the PCI sparse space encodings and the memory barrier requirement.
• ''arch/alpha/include/asm/special_insns.h'' for amask(). The core_apecs.h, core_lca.h and jensen.h headers were removed in Linux 6.10 by 430ad3f0ddd7 ("alpha: remove LCA and APECS based machines") and f81f335a56a8 ("alpha: remove DECpc AXP150 (Jensen) support"), and are available in earlier revisions.
• Arnd Bergmann, a4184174be36 ("alpha: drop pre-EV56 support"), Linux 6.10.
• Architecture Mask (amask).

See Mailing Lists for archive locations.