- Posts: 83
newlib (stdc) + uart + petit_fat integration
- wuselfuzz
-
Topic Author
- Offline
Less
More
02 Aug 2012 21:52 - 04 Aug 2012 09:37 #856
by wuselfuzz
newlib (stdc) + uart + petit_fat integration was created by wuselfuzz
Just some semi-personal notes for stuff to look into.
- Crash after updating the firmware, probably some issue with the FAT filesystem when the size of the image changes. Controller won't enter USB storage mode or main mode. Doesn't even turn off, complete firmware freeze. #defineable watchdog reset would be nice for not being forced to pull the battery cable. (powering the transmitter from a lab supply through the charge port works, no battery pulling). Seems like USB_Init() freezes...
... added debug-output to USB_Init(), now it USB Storage works again. Wtf?
- Corruption of model1.ini when changing model parameters and switching off the controller. Again, probably FAT issues.
- Throttle channel goes from -100% to 0% to -100% again. Other channels don't show this behaviour. "value" in channels.s/CHAN_ReadInput() is probably better off being s32 instead of u16.
- There might be an issue when writing an odd number of bytes to the FS (SPI_WriteFlash expecting an even number of bytes)
- Crash after updating the firmware, probably some issue with the FAT filesystem when the size of the image changes. Controller won't enter USB storage mode or main mode. Doesn't even turn off, complete firmware freeze. #defineable watchdog reset would be nice for not being forced to pull the battery cable. (powering the transmitter from a lab supply through the charge port works, no battery pulling). Seems like USB_Init() freezes...
... added debug-output to USB_Init(), now it USB Storage works again. Wtf?
- Corruption of model1.ini when changing model parameters and switching off the controller. Again, probably FAT issues.
- Throttle channel goes from -100% to 0% to -100% again. Other channels don't show this behaviour. "value" in channels.s/CHAN_ReadInput() is probably better off being s32 instead of u16.
- There might be an issue when writing an odd number of bytes to the FS (SPI_WriteFlash expecting an even number of bytes)
Last edit: 04 Aug 2012 09:37 by wuselfuzz.
Please Log in or Create an account to join the conversation.
- wuselfuzz
-
- Offline
Less
More
- Posts: 83
02 Aug 2012 23:56 #857
by wuselfuzz
Replied by wuselfuzz on topic Bugs
Everyone, there's definitely something weird going on.
I made a minor change to src/target/devo8/channels.c, flashed the fw, and it crashed in USB_init, console output stopped at "CYRF Mfg Data : '6A 2F 8A 4F B9 FF' "
Then, I changed "Open Whatever" to "Deviation" in SignOn() and it didn't crash anymore, but went to Wait Release/Released/Wait Release and my PC tried to mount the USB storage.
Changing the string resulted in a difference in code size (it's padded to 8 byte boundaries, the working version was 16 byte aligned).
I already tried changing the RAM size to 40k in the linker script and moving the RAM area to the end of physical RAM didn't change anything.
This looks like some kind of weird memory corruption.
Now doesn't newlib printf need a heap initialized? Is this done by libopencm3?
I made a minor change to src/target/devo8/channels.c, flashed the fw, and it crashed in USB_init, console output stopped at "CYRF Mfg Data : '6A 2F 8A 4F B9 FF' "
Then, I changed "Open Whatever" to "Deviation" in SignOn() and it didn't crash anymore, but went to Wait Release/Released/Wait Release and my PC tried to mount the USB storage.
Changing the string resulted in a difference in code size (it's padded to 8 byte boundaries, the working version was 16 byte aligned).
I already tried changing the RAM size to 40k in the linker script and moving the RAM area to the end of physical RAM didn't change anything.
This looks like some kind of weird memory corruption.
Now doesn't newlib printf need a heap initialized? Is this done by libopencm3?
Please Log in or Create an account to join the conversation.
- PhracturedBlue
-
- Offline
Less
More
- Posts: 4402
03 Aug 2012 01:19 #858
by PhracturedBlue
Replied by PhracturedBlue on topic Bugs
we have a custom printf implementation that is not very reliable.
if you #define printf to a NOP you may see things improve. This is a known issue that I've never been able to get to the bottom of. printfs in the FS code cause the system to hang all the time. Since I'm not currently focused on running on the actual Transmitter, I haven't spent anytime trying to stabilize there recently.
if you #define printf to a NOP you may see things improve. This is a known issue that I've never been able to get to the bottom of. printfs in the FS code cause the system to hang all the time. Since I'm not currently focused on running on the actual Transmitter, I haven't spent anytime trying to stabilize there recently.
Please Log in or Create an account to join the conversation.
- wuselfuzz
-
- Offline
Less
More
- Posts: 83
03 Aug 2012 06:57 #859
by wuselfuzz
Replied by wuselfuzz on topic Bugs
Ok. I'll try to find out what's going wrong there then. I'm not really the GUI designer type of person anyways. 
Please Log in or Create an account to join the conversation.
- PhracturedBlue
-
- Offline
Less
More
- Posts: 4402
03 Aug 2012 15:02 #864
by PhracturedBlue
Replied by PhracturedBlue on topic Bugs
FYI, This is the last build that I'm sure worked was 135983e0d80c. That was quite a while ago, however most of the work since then was GUI related. The core code in /target/devo8 has changed very little, so it shouldn't be too hard to get things working again.
I have family in town, so I may not have much time to work on it for the next few days.
I have family in town, so I may not have much time to work on it for the next few days.
Please Log in or Create an account to join the conversation.
- wuselfuzz
-
- Offline
Less
More
- Posts: 83
03 Aug 2012 20:20 #865
by wuselfuzz
Replied by wuselfuzz on topic Bugs
I had a few builds already working. I'm currently looking into that printf-issues when I find a bit of time again - me and my girlfriend just bought a house today (or rather signed a contract for building one). =)
Please Log in or Create an account to join the conversation.
- wuselfuzz
-
- Offline
Less
More
- Posts: 83
03 Aug 2012 21:33 #866
by wuselfuzz
Replied by wuselfuzz on topic Bugs
.plan:
- remove simple printf, I think possible printf/fprintf/vfprintf mixups might be the reason for the memory corruption.
- remove -lnosys for devo8 target. This only implements NOP stubs for newlib to get rid of linker errors.
- implement syscall-stubs for newlib to map filehandle 0-2 to uart and 3-n to open files. Can we live with a single open petit_fat file at a single time? If not, how many open files are required?
Have to catch up some sleep, getting at it tomorrow.
- remove simple printf, I think possible printf/fprintf/vfprintf mixups might be the reason for the memory corruption.
- remove -lnosys for devo8 target. This only implements NOP stubs for newlib to get rid of linker errors.
- implement syscall-stubs for newlib to map filehandle 0-2 to uart and 3-n to open files. Can we live with a single open petit_fat file at a single time? If not, how many open files are required?
Have to catch up some sleep, getting at it tomorrow.
Please Log in or Create an account to join the conversation.
- PhracturedBlue
-
- Offline
Less
More
- Posts: 4402
03 Aug 2012 22:17 #867
by PhracturedBlue
Anyhow, you plan sounds fine. I'm not really familiar with the bsp. Would that implement fgets as well? I had to do some nasty stuff in ini.c to use fread instead of fgets. The problem is that fgets seems to need to use fgetc, which is incredibly inefficient in petit_fat.
Replied by PhracturedBlue on topic Bugs
It is possible, but sprintf has been rock solid for us.wuselfuzz wrote: .plan:
- remove simple printf, I think possible printf/fprintf/vfprintf mixups might be the reason for the memory corruption.
currently we only support one fetit_fat file at a time. I was considering keeping the background BMP open, but testing so far shows that the performance penalty for opening a file isn't too great.- remove -lnosys for devo8 target. This only implements NOP stubs for newlib to get rid of linker errors.
- implement syscall-stubs for newlib to map filehandle 0-2 to uart and 3-n to open files. Can we live with a single open petit_fat file at a single time? If not, how many open files are required?
Have to catch up some sleep, getting at it tomorrow.
Anyhow, you plan sounds fine. I'm not really familiar with the bsp. Would that implement fgets as well? I had to do some nasty stuff in ini.c to use fread instead of fgets. The problem is that fgets seems to need to use fgetc, which is incredibly inefficient in petit_fat.
Please Log in or Create an account to join the conversation.
- PhracturedBlue
-
- Offline
Less
More
- Posts: 4402
04 Aug 2012 05:41 #868
by PhracturedBlue
* fixed write to support odd # of bytes and odd starting address
* erase sector before starting write
But I still get corruption. I am no longer sure if it is due to fprintf or the petit_fat/spi_write stuff. I need to do some simple text-cases to see, but I'm out of time for tonight.
The code to resize the file to 4095 bytes DOES work though, which is good
Replied by PhracturedBlue on topic Bugs
I did the following:wuselfuzz wrote: - Corruption of model1.ini when changing model parameters and switching off the controller. Again, probably FAT issues.
* fixed write to support odd # of bytes and odd starting address
* erase sector before starting write
But I still get corruption. I am no longer sure if it is due to fprintf or the petit_fat/spi_write stuff. I need to do some simple text-cases to see, but I'm out of time for tonight.
The code to resize the file to 4095 bytes DOES work though, which is good
That isn't necessary. the ADCs are only 12bits. They can't overflow a 16bit unsigned value. In this case it was just a typo in the calibration value (tx.ini). Once corrected, it worked as expected. there probably should be some limits on the calibration values to make sure they can't overflow a 16bit value though.- Throttle channel goes from -100% to 0% to -100% again. Other channels don't show this behavior. "value" in channels.s/CHAN_ReadInput() is probably better off being s32 instead of u16.
Please Log in or Create an account to join the conversation.
- wuselfuzz
-
- Offline
Less
More
- Posts: 83
04 Aug 2012 08:48 #869
by wuselfuzz
Keeping the values at 16 bit on a 32 bit cpu doesn't give a performance advantage usually. It just saves two bytes of ram. In this regards, I'd rather keep stuff in signed 32 bit values instead of running into dropped signs or integer overflows.
I have to investigate the fgets/fgetc stuff. In any way, I'd make it a priority that the foundation of the firmware will become rock-solid, it's a transmitter for RC planes after all.
Replied by wuselfuzz on topic Bugs
PhracturedBlue wrote: That isn't necessary. the ADCs are only 12bits. They can't overflow a 16bit unsigned value. In this case it was just a typo in the calibration value (tx.ini). Once corrected, it worked as expected. there probably should be some limits on the calibration values to make sure they can't overflow a 16bit value though.
Keeping the values at 16 bit on a 32 bit cpu doesn't give a performance advantage usually. It just saves two bytes of ram. In this regards, I'd rather keep stuff in signed 32 bit values instead of running into dropped signs or integer overflows.
I have to investigate the fgets/fgetc stuff. In any way, I'd make it a priority that the foundation of the firmware will become rock-solid, it's a transmitter for RC planes after all.
Please Log in or Create an account to join the conversation.
- wuselfuzz
-
- Offline
Less
More
- Posts: 83
04 Aug 2012 09:35 - 04 Aug 2012 09:37 #870
by wuselfuzz
Replied by wuselfuzz on topic Bugs
Nice related read:
neptune.billgatliff.com/newlib.html
/edit: Changed thread title to better reflect what's going on here.
/edit: Changed thread title to better reflect what's going on here.
Last edit: 04 Aug 2012 09:37 by wuselfuzz.
Please Log in or Create an account to join the conversation.
- PhracturedBlue
-
- Offline
Less
More
- Posts: 4402
04 Aug 2012 13:06 #871
by PhracturedBlue
And note that it is a signed 16bit value that we use for the channel data. the u16 is just for reading the ADC, and sure it could be 32bit but again, it wouldn't make any difference.
Replied by PhracturedBlue on topic Bugs
I understand there is no performance gain, however I have a different approach. Deviation is currently being targeted at Devo hardware, but I want it to be easily ported to other platforms (like the 9x) which have much more limited resources. So I believe in keeping the RAM allocation to an absolute minimum in the core code wherever possible. It may end up being impossible to port to a platform like the 9x in the end, but the philosophy remains. Sure this means we need to be careful not to overflow a 16bit value, but I've already tried to make that case. There are only very limited situations in which the value can actually exceed 16bits anyway.wuselfuzz wrote: Keeping the values at 16 bit on a 32 bit cpu doesn't give a performance advantage usually. It just saves two bytes of ram. In this regards, I'd rather keep stuff in signed 32 bit values instead of running into dropped signs or integer overflows.
And note that it is a signed 16bit value that we use for the channel data. the u16 is just for reading the ADC, and sure it could be 32bit but again, it wouldn't make any difference.
Please Log in or Create an account to join the conversation.
- PhracturedBlue
-
- Offline
Less
More
- Posts: 4402
04 Aug 2012 13:08 #872
by PhracturedBlue
Replied by PhracturedBlue on topic Bugs
rch4x0r did all the work with printf and friends. I know he spent quite a bit of time trying to get newlib to work before resorting to a custom printf solution. So I'll definitely be interested to see what you can come up with.
Please Log in or Create an account to join the conversation.
- PhracturedBlue
-
- Offline
Less
More
- Posts: 4402
04 Aug 2012 13:18 #873
by PhracturedBlue
Replied by PhracturedBlue on topic Bugs
Also, it may end up being better to give up on petit_fat. It doesn't look like we're under much RAM pressure at the moment, so the extra 4kb wouldn't hurt anything.The 2 big benefits would be that we could properly resize files (rather than the trick I'm using at the moment), and we could create new ones. Ive been reluctant to switch because I finally got this working, and didn't want to start over again.
Please Log in or Create an account to join the conversation.
- wuselfuzz
-
- Offline
Less
More
- Posts: 83
04 Aug 2012 16:20 #874
by wuselfuzz
Replied by wuselfuzz on topic newlib (stdc) + uart + petit_fat integration
newlib doesn't offer a filesystem. It just offers a standard C library, where the stdio file operations are just stubs.
Implementing _write_r that sends all written bytes to the uart already results in a working printf to console. I had this working in a sam7x project and in deviation, too. In my current local copy, file i/o doesn't work at all, but deviation boots up with debug output.
Implementing _write_r that sends all written bytes to the uart already results in a working printf to console. I had this working in a sam7x project and in deviation, too. In my current local copy, file i/o doesn't work at all, but deviation boots up with debug output.
Please Log in or Create an account to join the conversation.
- wuselfuzz
-
- Offline
Less
More
- Posts: 83
04 Aug 2012 17:41 #875
by wuselfuzz
Why does the main loop read media/devo8.bmp repeatedly?
Replied by wuselfuzz on topic newlib (stdc) + uart + petit_fat integration
Deviation
BootLoader : 'DEVO-08'
Power : '100mW'
SPI Flash : 'BF4ABF4A'
CYRF Mfg Data : '6A 2F 8A 4F B9 FF'
_open_r: pf_open (tx.ini) ok
_open_r: pf_open (media/config.ini) ok
_open_r: pf_open (media/sound.ini) ok
_open_r: pf_open (models/model1.ini) ok
_open_r: pf_open (media/heli.bmp) ok
_open_r: pf_open (media/txpower.bmp) ok
_open_r: pf_open (media/devo8.bmp) ok
_open_r: pf_open (media/options.bmp) ok
_open_r: pf_open (media/heli.bmp) ok
_open_r: pf_open (media/txpower.bmp) ok
_open_r: pf_open (media/devo8.bmp) ok
_open_r: pf_open (media/devo8.bmp) ok
_open_r: pf_open (media/devo8.bmp) ok
_open_r: pf_open (media/devo8.bmp) ok
_open_r: pf_open (media/devo8.bmp) ok
_open_r: pf_open (media/devo8.bmp) ok
_open_r: pf_open (media/devo8.bmp) ok
_open_r: pf_open (media/devo8.bmp) ok
_open_r: pf_open (media/devo8.bmp) ok
_open_r: pf_open (media/devo8.bmp) ok
_open_r: pf_open (media/devo8.bmp) okWhy does the main loop read media/devo8.bmp repeatedly?
Please Log in or Create an account to join the conversation.
- wuselfuzz
-
- Offline
Less
More
- Posts: 83
04 Aug 2012 18:41 #876
by wuselfuzz
Replied by wuselfuzz on topic newlib (stdc) + uart + petit_fat integration
Yikes! Pulled the watchdog stuff and it's resetting much too fast with all that filesystem debugging.
Please Log in or Create an account to join the conversation.
- wuselfuzz
-
- Offline
Less
More
- Posts: 83
04 Aug 2012 18:49 #877
by wuselfuzz
Replied by wuselfuzz on topic newlib (stdc) + uart + petit_fat integration
Ok, first version that runs with newlib stdio is up in my repo.
I'm pretty sure that file corruption occurs in petit_io, an early implementation of _write_r dumped model1.ini to UART (since I didn't check for the file descriptor to write to, everything written went to uart) and the dump I saw there was fine.
Now, with the added sector erase, the whole file is mostly zeroes.
Looking into that next.
It's no longer crashing on shutdown, btw. \o/
I'm pretty sure that file corruption occurs in petit_io, an early implementation of _write_r dumped model1.ini to UART (since I didn't check for the file descriptor to write to, everything written went to uart) and the dump I saw there was fine.
Now, with the added sector erase, the whole file is mostly zeroes.
Looking into that next.
It's no longer crashing on shutdown, btw. \o/
Please Log in or Create an account to join the conversation.
- wuselfuzz
-
- Offline
Less
More
- Posts: 83
04 Aug 2012 20:00 - 04 Aug 2012 20:14 #878
by wuselfuzz
Replied by wuselfuzz on topic newlib (stdc) + uart + petit_fat integration
Side note: the haxed ini.c avoiding fgets hangs in an endless loop after a file corruption.
/edit: I replaced ini.c with the original from google code. I can confirm that newlib fgetc/fgets are buffered, low level calls are using 1024 byte reads.
Now this happens:
model1.ini is still corrupted for testing, ignore that.
/edit: I replaced ini.c with the original from google code. I can confirm that newlib fgetc/fgets are buffered, low level calls are using 1024 byte reads.
Now this happens:
_open_r: pf_open (media/config.ini) ok
_read_r: len 1024, bytes_read 1024, result 0
Unknown font: font_color=000000
Unknown font: font_color=000000
Unknown font: font_color=000000
Unknown font: box_type=center
Unknown font: font_color=000000
Unknown font: box_type=center
Unknown font: box_type=center
Unknown font: bg_color=6B7380
Unknown font: bg_key1=E2E4E6
Unknown font: fg_key1=000000
Unknown font: bg_key2=8F95A1
Unknown font: fg_key2=000000
Unknown font: bg_key3=3270DF
Unknown font: fg_key3=FFFFFF
_read_r: len 1024, bytes_read 682, result 0
Unknown font: font_color=FFFFFF
Unknown font: font_color=000000
Unknown font: bg_color=FFFFFF
Unknown font: box_type=fill
Unknown font: font_color=FFFFFF
Unknown font: bg_color=FF0000
Unknown font: box_type=fill
Unknown font: font_color=FFFFFF
Unknown font: box_type=center
Unknown font: font_color=000000
Unknown font: bg_color=FFFFFF
Unknown font: box_type=fill
Unknown font: font_color=FFFFFF
Unknown font: bg_color=FF0000
Unknown font: box_type=fill
_read_r: len 1024, bytes_read 0, result 0
_close_r: file closed.
_open_r: pf_open (media/sound.ini) ok
_read_r: len 1024, bytes_read 219, result 0
_read_r: len 1024, bytes_read 0, result 0
_close_r: file closed.
_open_r: pf_open (models/model1.ini) ok
_read_r: len 1024, bytes_read 630, result 0
_read_r: len 1024, bytes_read 0, result 0
_close_r: file closed.
Failed to parse Model file: models/model1.inimodel1.ini is still corrupted for testing, ignore that.
Last edit: 04 Aug 2012 20:14 by wuselfuzz.
Please Log in or Create an account to join the conversation.
- wuselfuzz
-
- Offline
Less
More
- Posts: 83
04 Aug 2012 21:28 - 04 Aug 2012 21:59 #879
by wuselfuzz
Replied by wuselfuzz on topic newlib (stdc) + uart + petit_fat integration
Isolated the file corruption issue, working on a fix now.
Another issue: It seems, petit_fat doesn't update the file length in the FAT if you're writing to a file.
/edit: model1.ini now gets written properly. It turned out that SPIFlash_WriteBytes only wrote a single byte on a request with an uneven length. It's now doing a single byte write if the writeAddress is uneven, writes an even number of bytes using AAI and writes the leftover byte in a single write.
USB storage mode worked, because the msc2 stuff mostly did writes in 64 byte transfers.
I also changed the startup code a bit. It now works to exit the USB storage mode and go to the main loop when a valid FS is present on startup.
Another issue: It seems, petit_fat doesn't update the file length in the FAT if you're writing to a file.
/edit: model1.ini now gets written properly. It turned out that SPIFlash_WriteBytes only wrote a single byte on a request with an uneven length. It's now doing a single byte write if the writeAddress is uneven, writes an even number of bytes using AAI and writes the leftover byte in a single write.
USB storage mode worked, because the msc2 stuff mostly did writes in 64 byte transfers.
I also changed the startup code a bit. It now works to exit the USB storage mode and go to the main loop when a valid FS is present on startup.
Last edit: 04 Aug 2012 21:59 by wuselfuzz.
Please Log in or Create an account to join the conversation.
Time to create page: 0.084 seconds
-
Home
-
Forum
-
Development
-
Development
- newlib (stdc) + uart + petit_fat integration