ISP programming the GCHDMI board

Discussion in 'Modding and Hacking - Consoles and Electronics' started by Bearking, Dec 29, 2017.

  1. Bearking

    Bearking Konsolkongen

    Joined:
    Aug 2, 2010
    Messages:
    707
    Likes Received:
    62
    I could use some help programming the GCHDMI board. I have the board assembled, but as I don't have the exact ISP programmer specified in the guide, programming this thing turned out to be more of a hassle than I originally thought.

    https://github.com/citrus3000psi/GCVideo-Hardware-Designs/tree/master/Gamecube/DVI

    I have access to an Arduino Uno that I've used to ISP program other arduinos in the past, but that was much more straightforward than this is. I also have an USBtinyISP if that makes it any easier.

    I'm guessing I need to use something like avrdude or similar to program the flash on the board, but I have had very little success with that in the past, so I'm looking for someone to hopefully guide me on the right track.

    Another option is just to buy the specified programmer, but buying a new one every time just seems silly, and the delivery time could easily surpass a month, which I don't really want to wait for :)
     
  2. Zeigren

    Zeigren SuperSlim PS3s Are Not Supported

    Joined:
    Jan 19, 2014
    Messages:
    124
    Likes Received:
    45
    Oh yeah you should be able to use an Arduino UNO to program it using SPI. You're programming the flash memory so you wouldn't use something like AVRdude.

    This should work
    https://github.com/LowPowerLab/SPIFlash

    If that doesn't work you're looking for an SPI flashing program that can write to flash memory with 256 bytes per page.
     
    Bearking likes this.
  3. Bearking

    Bearking Konsolkongen

    Joined:
    Aug 2, 2010
    Messages:
    707
    Likes Received:
    62
    Thank you. Unfortunately I don't understand how to use it :/

    I have installed the library, but as far as I can see it can only choose to include it in a sketch. Not sure where to go from here, and unfortunately the documentation on the github page doesn't explain this.
     
  4. Zeigren

    Zeigren SuperSlim PS3s Are Not Supported

    Joined:
    Jan 19, 2014
    Messages:
    124
    Likes Received:
    45
    It looks like this library has a bit more support

    https://github.com/Marzogh/SPIFlash

    Adafruit has their own library actually, although it seems to be geared to this product and use case?
    https://learn.adafruit.com/adafruit...-circuit-python-circuitpython/using-spi-flash
    https://github.com/adafruit/Adafruit_SPIFlash

    This one actually has instructions on how to use it, so maybe try this one first
    https://github.com/nfd/spi-flash-programmer

    There's probably a better well documented easy solution out there, I just can't seem to find it for whatever reason.
     
    Bearking likes this.
  5. Bad_Ad84

    Bad_Ad84 Keyboard Error: Press F1 to Continue

    Joined:
    May 26, 2011
    Messages:
    7,941
    Likes Received:
    851
    If you get stuck, you can buy a flash chip and have it shipped to me. I'll program and forward it on
     
    Bearking likes this.
  6. Bearking

    Bearking Konsolkongen

    Joined:
    Aug 2, 2010
    Messages:
    707
    Likes Received:
    62
    Zeigren>
    Thank you again for pointing me in the right directions. I hope I have time to read through it tomorrow. I’ll let you know if I figure it out :)

    Bad_Ad84>
    Very kind of you to offer. I hope it doesn’t come to that, but I’ll throw you a PM if necessary :)
     
  7. Bearking

    Bearking Konsolkongen

    Joined:
    Aug 2, 2010
    Messages:
    707
    Likes Received:
    62
    Following your third link I installed Python 3 and pyserial on my Mac, so far so good. Now that I wanted to upload the sketch to the Arduino I get an error (see attached screenshot), which I can't figure out.

    The link in the sketch goes here:
    http://excamera.com/sphinx/article-crc.html

    But I have no idea what that's about.

    Decided to take a look at your fist link as well, but that was even more confusing to me :(
     

    Attached Files:

  8. Zeigren

    Zeigren SuperSlim PS3s Are Not Supported

    Joined:
    Jan 19, 2014
    Messages:
    124
    Likes Received:
    45
    Here try this, I fixed some random syntax stuff that the Arduino IDE didn't like

    Code:
    #include <avr/pgmspace.h>
    
    #define DATAOUT 11//MOSI
    #define DATAIN  12//MISO
    #define SPICLOCK  13//sck
    #define SLAVESELECT 10//ss
    
    //opcodes
    #define WREN  6
    #define WRDI  4
    #define RDSR  5
    #define WRSR  1
    #define READ  3
    #define WRITE 2
    #define SECTOR_ERASE 0x20
    #define CHIP_ERASE 0xC7
    
    byte buffer [256];
    
    // Via http://excamera.com/sphinx/article-crc.html
    static const PROGMEM uint32_t crc_table[16] = {
        0x00000000, 0x1db71064, 0x3b6e20c8, 0x26d930ac,
        0x76dc4190, 0x6b6b51f4, 0x4db26158, 0x5005713c,
        0xedb88320, 0xf00f9344, 0xd6d6a3e8, 0xcb61b38c,
        0x9b64c2b0, 0x86d3d2d4, 0xa00ae278, 0xbdbdf21c
    };
    
    unsigned long crc_update(unsigned long crc, byte data)
    {
        byte tbl_idx;
        tbl_idx = crc ^ (data >> (0 * 4));
        crc = pgm_read_dword_near(crc_table + (tbl_idx & 0x0f)) ^ (crc >> 4);
        tbl_idx = crc ^ (data >> (1 * 4));
        crc = pgm_read_dword_near(crc_table + (tbl_idx & 0x0f)) ^ (crc >> 4);
        return crc;
    }
    
    unsigned long crc_buffer(void)
    {
      unsigned long crc = ~0L;
      for(int i=0; i < 256; i++)
        crc = crc_update(crc, buffer[i]);
      crc = ~crc;
      return crc;
    }
    
    void setup()
    {
      Serial.begin(115200);
    
      pinMode(DATAOUT, OUTPUT);
      pinMode(DATAIN, INPUT);
      pinMode(SPICLOCK,OUTPUT);
      pinMode(SLAVESELECT,OUTPUT);
        
      digitalWrite(SLAVESELECT,HIGH); //disable device
      // SPCR = 01010000
      //interrupt disabled,spi enabled,msb 1st,master,clk low when idle,
      //sample on leading edge of clk,system clock/4 rate (fastest)
      //SPCR = (1<<SPE)|(1<<MSTR);
      SPCR = (1<<SPE)|(1<<MSTR)|(1<<SPI2X);
     
      byte clr;
      clr=SPSR;
      clr=SPDR;
     
      delay(10);
     
      buffer[0] = 0xca;
      buffer[1] = 0xfe;   
    }
    
    void loop()
    {
      byte addr1, addr2;
     
      while(Serial.available() == 0)
        ;
    
      int cmd = Serial.read();
     
      switch(cmd) {
      case '>':
        Serial.print('>');
        break;
      case 'e':
        erase_all();
        break;
      case 'r':
        addr1 = read_hex();
        addr2 = read_hex();
        read_page(addr1, addr2);
        break;
      case 'w':
        addr1 = read_hex();
        addr2 = read_hex();
        write_page(addr1, addr2);
        break;
      case 'd':
        dump_buffer();
        break;
      case 'c':
        dump_buffer_crc();
        break;
      case 'l':
        load_buffer();
        break;
      case 's':
        addr1 = read_hex();
        addr2 = read_nybble() << 4;
        erase_sector(addr1, addr2);
        break;
      case '?':
      case 'h':
        Serial.println("\nSPI Flash programmer");
        Serial.println(" e    : erase chip");
        Serial.println(" sXXX : erase 4k sector XXX (hex)");
        Serial.println(" c    : print buffer CRC-32");
        Serial.println(" rXXXX: read 256-byte page XXXX (hex) to buffer");
        Serial.println(" wXXXX: write buffer to 256-byte page XXXX (hex)");
        Serial.println(" d    : display the buffer (in hex)");
        Serial.println(" l<b> : load the buffer with <b>, where b is 256 bytes of data");
        Serial.println();
        Serial.println("Ex: r3700 - read 256 bytes from page 0x3700");
        Serial.println("Ex: lcafe[...]3737 - load the buffer with 256 bytes, first byte 0xca...");
      }
     
      Serial.flush();
    }
    
    void dump_buffer()
    {
      int counter;
     
      for(counter = 0; counter < 256; counter++) {
        Serial.print(buffer[counter] >> 4, HEX);
        Serial.print(buffer[counter] & 0xF, HEX);
      }
      Serial.println();
    }
    
    void dump_buffer_crc()
    {
      unsigned long crc = crc_buffer();
      Serial.print(crc_buffer(), HEX);
      Serial.println();
    }
    
    void load_buffer()
    {
      int counter;
      for(counter = 0; counter < 256; counter++) {
        buffer[counter] = read_hex();
      }
    }
    
    byte read_nybble()
    {
      int nybble;
      while((nybble = Serial.read()) == -1)
        ;
     
      if(nybble >= 'A') {
        // works for lowercase as well (but no range checking of course)
        return 9 + (nybble & 0x0f);
      } else {
        return nybble & 0x0f;
      }
    }
    
    byte read_hex()
    {
      byte val;
     
      val = (read_nybble() & 0xf) << 4;
      val |= read_nybble();
     
      return val;
    }
    
    byte spi_transfer(volatile char data)
    {
      SPDR = data;                    // Start the transmission
      while (!(SPSR & (1<<SPIF)))     // Wait for the end of the transmission
      {
      };
      return SPDR;                    // return the received byte
    }
    
    void read_page(byte adr1, byte adr2)
    {
      //READ EEPROM
      int counter;
      digitalWrite(SLAVESELECT,LOW);
      spi_transfer(READ);
      spi_transfer(adr1); // bits 23 to 16
      spi_transfer(adr2); // bits 15 to 8
      spi_transfer(0);    // bits 7 to 0
      for(counter = 0; counter < 256; counter++) {
        buffer[counter] = spi_transfer(0xff);
      }
      digitalWrite(SLAVESELECT,HIGH); //release chip, signal end transfer
    }
    
    void wait_for_write()
    {
      byte statreg = 0x1;
     
      while((statreg & 0x1) == 0x1) {
        // Wait for the chip.
        digitalWrite(SLAVESELECT, LOW);
        spi_transfer(RDSR);
        statreg = spi_transfer(RDSR);
        digitalWrite(SLAVESELECT, HIGH);
      } 
    }
    
    void write_page(byte adr1, byte adr2)
    {
      int counter;
     
      digitalWrite(SLAVESELECT,LOW);
      spi_transfer(WREN); //write enable
      digitalWrite(SLAVESELECT,HIGH);
      delay(10);
    
      digitalWrite(SLAVESELECT,LOW);
      spi_transfer(WRITE); //write instruction
      spi_transfer(adr1); // bits 23 to 16
      spi_transfer(adr2); // bits 15 to 8
      spi_transfer(0);    // bits 7 to 0
    
      for (counter = 0; counter < 256; counter++)
      {
        spi_transfer(buffer[counter]);
      }
      digitalWrite(SLAVESELECT,HIGH);
      delay(1);
     
      wait_for_write();
    }
    
    void erase_all()
    {
      int counter;
     
      digitalWrite(SLAVESELECT,LOW);
      spi_transfer(WREN); //write enable
      digitalWrite(SLAVESELECT,HIGH);
      delay(10);
    
      digitalWrite(SLAVESELECT,LOW);
      spi_transfer(CHIP_ERASE);
      digitalWrite(SLAVESELECT,HIGH);
      delay(1);
     
      wait_for_write();
    }
    
    void erase_sector(byte addr1, byte addr2)
    {
      digitalWrite(SLAVESELECT,LOW);
      spi_transfer(WREN);
      digitalWrite(SLAVESELECT,HIGH);
      delay(10);
     
      digitalWrite(SLAVESELECT,LOW);
      spi_transfer(SECTOR_ERASE);
      spi_transfer(addr1);
      spi_transfer(addr2);
      spi_transfer(0);
      digitalWrite(SLAVESELECT,HIGH);
    }
     
    Bearking likes this.
  9. Bearking

    Bearking Konsolkongen

    Joined:
    Aug 2, 2010
    Messages:
    707
    Likes Received:
    62
    Thank you. That’s very kind of you :)

    Been hungover all day, but I should have time to try this tomorrow :)

    Happy new year.
     
  10. Bearking

    Bearking Konsolkongen

    Joined:
    Aug 2, 2010
    Messages:
    707
    Likes Received:
    62
    Sorry that it took so long for me to get back. Your edited sketch uploads fine to my Arduino :)

    https://github.com/nfd/spi-flash-programmer

    Can I follow the example code to read, write and verify the flash? What I mean is, do I have to change this part:

    Code:
    --flash-offset 0 -s 4096 -f dump.bin
    
    EDIT: I realize I need to change "dump.bin" to the correct file name :)

    Also, I don't have "/dev/cu.usbserial" on my Mac. I've got something called "/dev/cu.usbmodem1411".

    Is this the Mac equivalent? It disappears when I remove the Arduino from the USB-port :)
     
    Last edited: Jan 7, 2018
  11. Bearking

    Bearking Konsolkongen

    Joined:
    Aug 2, 2010
    Messages:
    707
    Likes Received:
    62
    Okay tried the above with all wires correctly connected.

    I get this:
    Code:
    Martinbook:GCHDMI konsolkongen$ python3 spi_flash_programmer_client.py -d /dev/cu.usbmodem1411 --flash-offset 0 -s 4096 -f gcvideo-dvi-shuriken-v3-2.3.bin write
    Traceback (most recent call last):
      File "spi_flash_programmer_client.py", line 2, in <module>
        import serial
    ModuleNotFoundError: No module named 'serial'
     
  12. Zeigren

    Zeigren SuperSlim PS3s Are Not Supported

    Joined:
    Jan 19, 2014
    Messages:
    124
    Likes Received:
    45
    Looks like it can't find pyserial? I'm not sure how you installed it before but I'll outline how to install it real quick

    Grab pyserial-3.4.tar.gz from here (https://pypi.python.org/pypi/pyserial)
    On OSX open Terminal and cd into the folder you downloaded it to
    Unpack the file

    Code:
    tar -xzf pyserial-3.4.tar.gz
    Then cd into the pyserial-3.4 folder that was just created then install pyserial

    Code:
    sudo python3 setup.py install
    That should work
    In OSX I'm not sure you need to specify python3, unless you have python2 installed as well and are differentiating them that way.
     
    Bearking likes this.
  13. Bearking

    Bearking Konsolkongen

    Joined:
    Aug 2, 2010
    Messages:
    707
    Likes Received:
    62
    Lol I was actually quite sure I had already done that, and that the "serial" it was complaining about was the lack of was '/dev/cu.usbserial' :p

    That helped a lot. Now it's writing to the chip but once it's done I get this:

    Code:
    Martinbook:GCHDMI konsolkongen$ python3 spi_flash_programmer_client.py -d /dev/cu.usbmodem1411 --flash-offset 0 -s 4096 -f gcvideo-dvi-shuriken-v3-2.3.bin write
    Writing gcvideo-dvi-shuriken-v3-2.3.bin at 0
    475136 of 475392Traceback (most recent call last):
      File "spi_flash_programmer_client.py", line 223, in <module>
        main()
      File "spi_flash_programmer_client.py", line 214, in main
        chat.writeFile(args.filename, int(args.flash_offset))
      File "spi_flash_programmer_client.py", line 176, in writeFile
        assert len(pageData) == SECTOR_SIZE
    AssertionError
    Martinbook:GCHDMI konsolkongen$ python3 spi_flash_programmer_client.py -d /dev/cu.usbmodem1411 --flash-offset 0 -s 4096 -f gcvideo-dvi-shuriken-v3-2.3.bin verify
    Verifying gcvideo-dvi-shuriken-v3-2.3.bin with flash start 0 and file start 0
    475392Martinbook:GCHDMI konsolkongen$ 
     
  14. KaosEngineer

    KaosEngineer Spirited Member

    Joined:
    Jun 7, 2016
    Messages:
    161
    Likes Received:
    70
    The file being written is not an exact multiple of the sector size. 475,136 is 116 4K sectors but it's trying to write a bit more up to 475,392 bytes an extra 256 bytes not a full 4K sector.

    What size is gcvideo-dvi-shuriken-v3-2.3.bin?

    Update: I found that file. It is 149,554 bytes. It's not an even multiple of 4K byte sectors either. Not sure why it looks like it's trying to write 475,392 bytes instead of 149,554?
     
    Last edited: Jan 13, 2018
    Bearking likes this.
  15. Bearking

    Bearking Konsolkongen

    Joined:
    Aug 2, 2010
    Messages:
    707
    Likes Received:
    62
    Nope I have no idea what went wrong there either. You're right I was trying to write the wrong bin-file, although they were named the same... >_<

    Unfortunately I can't get it working again. Really haven't changed anything in the setup as I just left the computer as is was with the arduino connected :p

    EDIT: Does this look correct?

    Code:
    0Martinbook:GCHDMI konsolkongen$ python3 spi_flash_programmer_client.py -d /dev/cu.usbmodem1411 --flash-offet 0 -s 4096 -f gcvideo-dvi-shuriken-v3-2.3.bin write
    Writing gcvideo-dvi-shuriken-v3-2.3.bin at 0
    Martinbook:GCHDMI konsolkongen$ python3 spi_flash_programmer_client.py -d /dev/cu.usbmodem1411 --flash-offset 0 -s 4096 -f gcvideo-dvi-shuriken-v3-2.3.bin verify
    Verifying gcvideo-dvi-shuriken-v3-2.3.bin with flash start 0 and file start 0
    0Martinbook:GCHDMI konsolkongen$
    
     
    Last edited: Jan 13, 2018
  16. KaosEngineer

    KaosEngineer Spirited Member

    Joined:
    Jun 7, 2016
    Messages:
    161
    Likes Received:
    70
    My guess is the software writes 4K sectors and if there's not a full sector left the software pads it to fill an entire sector before writing the last little bit.

    But, why it won't write again, I haven't a clue. GL :)
     
    Bearking likes this.
  17. Bearking

    Bearking Konsolkongen

    Joined:
    Aug 2, 2010
    Messages:
    707
    Likes Received:
    62
    Thank you :)
     
  18. KaosEngineer

    KaosEngineer Spirited Member

    Joined:
    Jun 7, 2016
    Messages:
    161
    Likes Received:
    70
    I see this line executed "print("Verifying %s with flash start %d and file start %d" % (filename, flashStartByte, fileStartByte))"

    There were other numbers updating below this line weren't there? Like when verifying it was showing the addresses it was testing -- this line of code "sys.stdout.write('\r%d' % (idx))" -- should have printed 0, 256, 512, 768, 1024,...
    It would have remained on the same line as only a carriage return (\r) is printed no linefeed (\n).

    Carriage return (\r) moves the cursor to the beginning of the current line and any text output previously will be overwritten. If a linefeed is added the cursor will move down a line and the text printed on the next line. Thus, leaving behind many rows of printing:

    0
    256
    512
    768
    ...

    And, as there's no output below it showing mismatch errors during the verify operation by this line of code "print("Mismatch %x != %x, retrying" % (deviceCRC, fileCRC))" and no Exception raised (Mismatch), I'd say it verified correctly.
     
    Last edited: Jan 13, 2018
    Bearking likes this.
  19. Bearking

    Bearking Konsolkongen

    Joined:
    Aug 2, 2010
    Messages:
    707
    Likes Received:
    62
    At first it was done almost instantaneously, but now it takes about 10-15 minutes to write and verify. Is this correct?

    When it's doing this, I do get this counter that goes up until it reaches the full 4194304.

    Screen Shot 2018-01-14 at 15.46.29.png

    Verify takes just as long.

    TX is flashing the entire time on the Arduino when writing. As is RX when it's verifying.
     
  20. KaosEngineer

    KaosEngineer Spirited Member

    Joined:
    Jun 7, 2016
    Messages:
    161
    Likes Received:
    70
    Sounds correct. Not sure why the file you are flashing is 4MB's as the one I see at github is only 146KB. I guess it's counting bits flashed not bytes of the file written. As there are 4,096 Kbits or 4Mbits in this flash memory chip. But the file only has 1,196,416 bits to be flashed (0x24830 bytes * 8 bits/byte = 0x124180 bits).

    https://github.com/citrus3000psi/GC.../Gamecube/DVI/gcvideo-dvi-shuriken-v3-2.3.bin

    After flashing, install it. :D See if its working.

    Update: the chip being flashed is 4Mbits - FLASH - NOR Memory IC 4Mb (512K x 8) SPI 75MHz 8-SOP by Macronix part # MX25V4006EM1I-13G suggested as a drop-in replacement for the one listed in the BOM - Micron M25P40-VMN6PB.

    What baud rate are you talking to the arduino with? The speed may be from a slow communications rate - this flash can go up to 75MHz but with the Arduino it may be using 9600 baud. Will take a long time with such a S...L...O...W transfer speed.

    In Zeigren's post #8 above I see the serial port set to 115200 baud (bits/second) in the Arduino Sketch:

     
    Last edited: Jan 14, 2018 at 11:15 AM

Share This Page