Since the initial release, community contributions have pushed data efficiency from ~2.4x to 5.5x against modded-nanogpt, more than doubling in a few days. The key changes are: shuffling at the start of each epoch, which had outsized impact on multi-epoch training; learned projections for value embeddings instead of separate embedding tables; swapping squared ReLU for SwiGLU activation; and ensembling multiple models. 10x data efficiency seems reachable in the short term. 100x might be feasible by the end of the year, given how many directions remain unexplored, but it will require serious exploration on the algorithms side.
AND Rd, Rs1, Rs2
，更多细节参见PDF资料
Lex: FT’s flagship investment column
:first-child]:h-full [&:first-child]:w-full [&:first-child]:mb-0 [&:first-child]:rounded-[inherit] h-full w-full
相比之下，Windows Phone 更像一个过于礼貌的管家：它把信息递到你面前，然后安静地退到一旁。从商业角度看它注定是输家；但从设计价值看，它尊重用户的注意力，减少干扰，强调确定性和一致性。这种「数字谦逊」，在今天看来几乎是一种奢侈。