Benchmarking every TI-84 Plus CE OS! [5.0.0-5.6.0]

[thelastmillennial]

Hey, this is cross post from Cemetech, I thought you guys might find it interesting though!

As you all know, as technology gets updates with more features, that device typically starts slowing down. I was wondering, is TI doing this to our calculator's too? That's why I set up a series of tests and ran them from OS 5.0.0 to 5.4.0. Each one tests a specific function of the calculator. You can see a description of each test and the full code for each one down below.

Anyways, enjoy the chart: green is the best time and red is the worst time!

Each test was performed from the RAM, on the same calculator (revision pre-A), in the same order, and after a full mem clear. The calculator was plugged in and fully charged the entire time. Each test, with the exception of prgmRAYTRSPR was completed 5 times then the times were averaged into the table above. I only ran prgmRAYTRSPR once because it takes over 30 minutes per run and ain't nobody got time to do that 5 times over! I have yet to retest OSes under 5.3.0 for new prgmRAYTRSPR times.

Overall it seems if you want the best BASIC performance, then OS 5.0.0 or OS 5.3.1 is the way to go. However, if you just care about the speed of specific functions, then you can't go wrong with OS 5.1.x or 5.2.1.

I'd like to dig into the numbers a bit more though.

OS 5.0.0 is the champ for random numbers, graphing horizontal and vertical lines, and overall BASIC performance. Then OS 5.3.0 (relatively) crushed the times for the summation and list tests but fell behind a bit in BASIC performance.

As you can see by the graph, the only reason the newer OSes had good total scores was because the raytracing program was such a huge factor in this test. However, if you look at each function separately, it's clear they are generally getting slower on newer OSes.

TI changed something for the better between OS 5.0.0 and OS 5.1.0 that made list sorting almost 9x faster on newer OSes! Even for numbers that weren't 0s.

TI also changed something for the worse between OS 5.2.1 and OS 5.3.0 because every OS newer than 5.2.1 is typically 3 seconds slower.

Here are the tests I performed, you can get prgmTIMER here.
ABENCH (runs every benchmark and stores the results into a list)

Code: Select all

SetUpEditor L₆
9→dim(L₆
prgmT01RAND
Ans→L₆(1
prgmT02GRAPH
Ans→L₆(2
prgmT03GRAPH
Ans→L₆(3
prgmT04SUM
Ans→L₆(4
prgmT05LIST
Ans→L₆(5
prgmT06LIST
Ans→L₆(6
prgmT07LIST
Ans→L₆(7
L₆


T01RAND (random integer test)

Code: Select all

0→B
For(A,1,5
Asm(prgmTIMER
rand(100
Asm(prgmTIMER
Ans+B→B
End
B/5

T02GRAPH (Horizontal line test)

Code: Select all

0→W
For(A,1,5
Asm(prgmTIMER
ClrDraw
For(B,Ymax,Ymin,­Y
Horizontal B
End
Asm(prgmTIMER
Ans+W→W
End
W/5

T03GRAPH (vertical line test)

Code: Select all

0→W
For(A,1,5
Asm(prgmTIMER
ClrDraw
For(B,Xmax,Xmin,­X
Vertical B
End
Asm(prgmTIMER
Ans+W→W
End
W/5

T04SUM (summation test)

Code: Select all

0→W
For(A,1,5
Asm(prgmTIMER
Σ(X²,X,0,1000
Asm(prgmTIMER
Ans+W→W
End
W/5

T05LIST (creating list test)

Code: Select all

0→W
For(A,1,5
DelVar L₁
Asm(prgmTIMER
999→dim(L₁
Asm(prgmTIMER
Ans+W→W
End
W/5

T06LIST (sorting list of only 0s)

Code: Select all

0→W
For(A,1,5
DelVar L₁
100→dim(L₁
Asm(prgmTIMER
SortA(L₁
Asm(prgmTIMER
Ans+W→W
End
W/5

T07LIST (sorting list of only 7s)

Code: Select all

0→W
For(A,1,5
DelVar L₁
100→dim(L₁
Fill(7,L₁
Asm(prgmTIMER
SortA(L₁
Asm(prgmTIMER
Ans+W→W
End
W/5

RAYTRSPR ( this is a 3D modeling test using raycasting) (I didn't make this program, this was made by TheScienceElf, you can view his video here.)

Code: Select all

Asm(prgmTIMER
ClrDraw:1→G
{12,24,23,22,21,20}→L₁

For(Y,0,162,2G)
(81-Y)/164→B
For(X,0,262,2G)
(X-131)/164→A

A²+B²+1→E:­2B-12→F
34.75→H:F²-4EH→D

­1/B→T

AT→U:T→V
remainder(abs(int(U)+int(V)),2)→C

12→L:12→M
12-(T>0)(C+1)→C

If D≥0:Then
(­F-√(D))/(2E)→T
AT→I:BT-1→J:6-T→K
J/1.5→M

If M>0:Then
int(10M)→M
L₁(1+int(M/2))→L
L₁(1+int(M/2+.5))→M
Else:12→M
End

{I,J,K}/1.5→L₃
{A,B,1}→L₂
2sum(L₃L₂)→S

SL₃-L₂→L₄
L₄(1)→P:L₄(2)→Q:L₄(3)→R

­(J+2)/Q→T

If T>0:Then
PT+I→U:RT+6-K→V
11-remainder(abs(int(U)+int(V)),2)→C
Else:12→C:End

Else:If T>0:Then
1→E:­4→F
U²+V²-12V+37.75→H
F²-4EH→D

12(D≥0)+C(D<0)→L
L→M
End:End

Pxl-On(Y,X,C)
Pxl-On(Y+1,X,L)
Pxl-On(Y,X+1,M)
Pxl-On(Y+1,X+1,C)
End:End
Asm(prgmTIMER
Ans

So there you go, personally I think OS 5.3.0 has the best performance to match its features (Asm84CEprgm token, yay!). Let me know if you have any questions or concerns!

[thelastmillennial]

OK, a heck of a lot has changed since last I did this test, so I decided to go back and do all these tests over again on my Pre-A TI-84 Plus CE. I made a few changes so the graph won't line up completely with last test's results.

Specifically I changed:

• I added a few more OSes to test. 5.2.0, 5.2.2, 5.4.1, 5.5.5, and 5.6.0
• T06LIST to SortA( a randomly generated list with the seed 1->rand.
• T07LIST to SortD( a randomly generated list with the seed 1->rand.
• I added T08ASM which tests how long it took for the timer program to launch. This measures the speed overhead it took for the program to start and return.
• I removed the ray trace benchmark for now because it just takes too long to test.
• I added an extra row that shows how much faster a certain OS was over the baseline (the slowest OS).

You can download the whole benchmark group here.

Enough talk, let's see the numbers!
Green is the best time and Red is the worst time

Each test was performed from the RAM, on the same calculator, in the same order, and after a full mem clear. The calculator was plugged in and fully charged the entire time. Each test was completed 5 times, then the times were averaged into the table above. arTIfiCE and AsmHOOK were only installed on OS versions that required them; all jail breaking utilities were stored in the Archive.

OK, we see a lot of similarities to last years results. OS 5.0.0 continues to be the slowest at sorting but it's generally pretty darn fast at everything else.

As you get to the newer OSes which added removed features, it's clear that the calculator is getting slower and slower at doing every function. Only by about 1% from OS 5.3.0, but if we see another drop in performance like OS 5.3.0 brought for the graphing tests, there could be significant speed issues for those with a pre-M TI-84 Plus CE.

One result I was surprised to see was that OS 5.5+ was negligibly slower at launching ASM programs than OSes that supported ASM. I figured TI would have done more to block ASM programs from launching.

So those are the results for this year! I'm probably going to stop re-testing every single OS every time a new OS gets released because this took 4 hours to organize, test, collect data, and post. I'll just be testing the newest OS and tacking it onto the end of my Excel sheet. Let me know what you thought of the results! Anything you guys want me to test next time? Are there any results that surprised you?

[thelastmillennial]

I received some criticism on my last table but thanks to the help of KermMartian and LogicalJoe, I've created a new table that's actually useful and doesn't exaggerate the color of a negligible difference! Not only that, but I'm sharing my results so you can mess with the data however you'd like! Go take a look at it here: https://1drv.ms/x/s!Al_ZukYl7XyyyzQRzhP ... 4?e=6fXiGA!

This table compares the speed of each OS to OS 5.0.0 (which I used as the baseline):


By the request of Adriweb, I also made a table that compares the speed of each OS over the previous OS:


My previous conclusion needs some adjusting. Although new OS versions are still generally getting slower, most of the time the difference per generation is negligible and you'll never notice it. The only exception to this is OS 5.1.0 significantly speeding up sorting lists by about 4 times! Along with OS 5.3.0 slowing down graphing by almost 14%.

So that's the new table format! Thanks again to those who helped me figure out Excel and requested data!

[Adriweb]

Thank you, great work