Threaded Mode | Linear Mode

ZiNgA BuRgA · 10/04/2015 06:43 PM

You're mostly correct in your understanding.

Rcomage doesn't have a GIM parser yet, so it can't figure out what settings to use.  It currently just makes a guess from what the RCO file is - it doesn't know anything about the GIM.

A possible solution is as you mention - parse the GIM file and write the settings to the XML file.  Then the compile process would need to read in the settings and generate the correct GIM file.

A limitation of that though, is that conversions may not work so nicely as GimConv doesn't support all of them.
Also, it may be possible to swap types - at least on the PSP, where most (all maybe?) firmware GIMs are 8-bit (indexed), but can be replaced by 32-bit (8888) and they work fine (provided you don't hit the internal memory limitations).  I don't have a PS3 to test with, so I don't know what GIM types could be swapped.

---

The UMDFlag and minFirmwareVer attributes, I don't really know about.  They are values in the RCO file which seem to correspond with versioning, but I don't fully understand them.  'minFirmwareVer' was used because I map the values in the RCO file to PSP firmware versions:

C Code

		switch(rco->verId) {
			case 0x70: fputs("1.0", fp); break;
			case 0x71: fputs("1.5", fp); break;
			case 0x90: fputs("2.6", fp); break;
			case 0x95: fputs("2.7", fp); break;
			case 0x96: fputs("2.8", fp); break;
			case 0x100: fputs("3.5", fp); break;
			//case 0x107: fputs("ps3", fp); break;
			default: fprintf(fp, "unknownId0x%x", rco->verId);
		}

So I've seen a 0x107 for PS3 RCOs, I suppose some may be 0x130 and maybe something else. It may correspond with a particular firmware version, I don't know.

sandungas · 12/04/2015 02:02 AM

(10/04/2015 06:43 PM)ZiNgA BuRgA Wrote: You're mostly correct in your understanding.

Rcomage doesn't have a GIM parser yet, so it can't figure out what settings to use. It currently just makes a guess from what the RCO file is - it doesn't know anything about the GIM.

A possible solution is as you mention - parse the GIM file and write the settings to the XML file. Then the compile process would need to read in the settings and generate the correct GIM file.

A limitation of that though, is that conversions may not work so nicely as GimConv doesn't support all of them.

I think the addition of the gim type to the rcoxml is good enought by now because allows the user to modify the gim types for each image manually and then compile with rcogui from PNG--»RCO
For the gim types you can use the same gim settings you have been using in previous versions, this way the image conversion is the same than previous versions but is more flexible (and fixes partially the problem with PNG--»RCO conversions for PS3)

I made a list of options for gimconv.cfg with all the gim formats used by PSP and PS3 using different names, take a look if is correct

C Code

//--------------------------------------------------------
//  Rcomage options
//--------------------------------------------------------

	//  GIM formats used by OFW RCO's

option -pspbpp8 {				// First most used GIM format in PSP
	format_style = psp
	format_endian = little
	image_format = index8
	pixel_order = faster
	limit_image_width = 512
	limit_image_height = 512
	output_sequence = off
}
	
option -ps3bpp32 {				// First most used GIM format in PS3
	format_style = psp
	format_endian = big
	image_format = rgba8888
	pixel_order = normal
}

option -ps3dxt5 {				// Second most used GIM format in PS3
	format_style = psp
	format_endian = big
	image_format = dxt5
}

	//  GIM formats supported by CFW RCO's

option -pspbpp4 {
	format_style = psp
	format_endian = little
	image_format = index4
	pixel_order = faster
	limit_image_width = 512
	limit_image_height = 512
	output_sequence = off
}

option -pspbpp16 {
	format_style = psp
	format_endian = little
	image_format = rgba5551
	pixel_order = faster
	limit_image_width = 512
	limit_image_height = 512
	output_sequence = off
}

option -pspbpp16a {
	format_style = psp
	format_endian = little
	image_format = rgba4444
	pixel_order = faster
	limit_image_width = 512
	limit_image_height = 512
	output_sequence = off
}

option -pspbpp16p {
	format_style = psp
	format_endian = little
	image_format = rgba5650
	pixel_order = faster
	limit_image_width = 512
	limit_image_height = 512
	output_sequence = off
}

option -pspbpp32 {
	format_style = psp
	format_endian = little
	image_format = rgba8888
	pixel_order = faster
	limit_image_width = 512
	limit_image_height = 512
	output_sequence = off
}

This completes all ?, in some way are making uneeded the "convenient options"
Note are divided in 2 groups, the GIM formats used in OFW are placed at top to give them some kind of preference, because the "First most used GIM format in OFW" for each console can be considered the default
This way is posible to use them from the rcoxml:

For PSP the default image type="pspbpp8"
The user can replace the type manually by: pspbpp4, pspbpp16a, pspbpp16p, pspbpp32 (to change image quality)

XML Code

<RcoFile UMDFlag="0" rcomageXmlVer="1.1" type="psp" minFirmwareVer="1.5">
	<MainTree name="sysconf_plugin_about">
		<ImageTree>
			<Image name="tex_about_psp_WW_0" src="Images\tex_about_psp_WW_0.png" format="gim" type="pspbpp8" compression="zlib" unknownByte="0" />

For PS3 the default image type="ps3bpp32"
The user can replace the type manually by: ps3dxt5 (for the images that needs it)

XML Code

<RcoFile UMDFlag="0" rcomageXmlVer="1.1" type="ps3" minFirmwareVer="unknownId0x130">
	<MainTree name="xmb_plugin">
		<ImageTree>
			<Image name="tex_loading_icon" src="Images\tex_loading_icon.png" format="gim" type="ps3bpp32" compression="zlib" unknownByte="0" />

(10/04/2015 06:43 PM)ZiNgA BuRgA Wrote: Also, it may be possible to swap types - at least on the PSP, where most (all maybe?) firmware GIMs are 8-bit (indexed), but can be replaced by 32-bit (8888) and they work fine (provided you don't hit the internal memory limitations). I don't have a PS3 to test with, so I don't know what GIM types could be swapped.

For PSP i grouped them as "GIM formats supported by CFW RCO's"
For PS3 i don't know if there are alternatives, i guess yes but because the default is rgba8888 the others are lower quality so i think not much people will have interest in reducing quality

(10/04/2015 06:43 PM)ZiNgA BuRgA Wrote: The UMDFlag and minFirmwareVer attributes, I don't really know about. They are values in the RCO file which seem to correspond with versioning, but I don't fully understand them. 'minFirmwareVer' was used because I map the values in the RCO file to PSP firmware versions:
C Code
1 2 3 4 5 6 7 8 9 10
		switch(rco->verId) {
			case 0x70: fputs("1.0", fp); break;
			case 0x71: fputs("1.5", fp); break;
			case 0x90: fputs("2.6", fp); break;
			case 0x95: fputs("2.7", fp); break;
			case 0x96: fputs("2.8", fp); break;
			case 0x100: fputs("3.5", fp); break;
			//case 0x107: fputs("ps3", fp); break;
			default: fprintf(fp, "unknownId0x%x", rco->verId);
		}
So I've seen a 0x107 for PS3 RCOs, I suppose some may be 0x130 and maybe something else. It may correspond with a particular firmware version, I don't know.

I was curious about what is the UMDFlag, i know part of the history you found it used in some rco's that was in a UMD but i don't know more, one of the doubts i have is if posible to use rco's in PS3 bluray discs and incase are supported if uses the same flag
I have no idea, but as a precaution you could change the name to "DiscFlag"... the meaning is almost the same because UMD and BluRay are "disc media type"
But changing the name of something without being sure of his purpose is a bit risky so not so sure what i would do

For minFirmwareVer im not sure what to say, if it changes so much (and increases) it needs to be some kind of versioning, but i can't imagine
I see you had 0x107 from a old ps3 rco, there is another one posted in wiki with 0x120, and the example i posted here is 0x130... it seems are still increasing
By now for the unknowns you can change this line to output only the hex value, it will look better not like a bug

C Code

			default: fprintf(fp, "0x%x", rco->verId);

ZiNgA BuRgA · 14/04/2015 04:43 AM

(12/04/2015 02:02 AM)sandungas Wrote: I was curious about what is the UMDFlag, i know part of the history you found it used in some rco's that was in a UMD but i don't know more, one of the doubts i have is if posible to use rco's in PS3 bluray discs and incase are supported if uses the same flag

PS3 Blu-rays? You mean games?
I've never seen RCOs in PSP games. By UMD, I'm referring to UMD videos, where resources for the menu come from an RCO. (For your information: Sony tried to replace DVDs with their UMD video disc format)
The BD video format is fairly standard and doesn't have RCOs, so you won't find them there.

(12/04/2015 02:02 AM)sandungas Wrote: By now for the unknowns you can change this line to output only the hex value, it will look better not like a bug
C Code
			default: fprintf(fp, "0x%x", rco->verId);

I'd prefer to explicitly state that it's an unknown ID though, otherwise it could get confusing with how some other numbers are represented in the XML (floats are numbers whilst integers are prefixed with 0x).

Thanks for your help!

sandungas · 17/04/2015 09:36 AM

(14/04/2015 04:43 AM)ZiNgA BuRgA Wrote: PS3 Blu-rays? You mean games?
I've never seen RCOs in PSP games. By UMD, I'm referring to UMD videos, where resources for the menu come from an RCO. (For your information: Sony tried to replace DVDs with their UMD video disc format)
The BD video format is fairly standard and doesn't have RCOs, so you won't find them there.

In "PS3 BD games" they uses a file in the root of the disc named http://www.psdevwiki.com/ps3/PS3_DISC.SFB
Basically, is used to trigger on/off some features and to identify the contents of the disc, this way the firmware knows what to do with the content
I found it used too in some old "BD video" published by sony (if i remember correctly, in a reedition of "top gun" and "the mariachi" films)

What im going to say is speculative, but i think sony prepared some rare formats that later was not used, it seems there is a "BD music" and "BD photo"... and well... because they could easilly implement support in ps3 to continue with what they was doing in psp using umd's with rco's
I always thought there are some undocumented disc formats for PS3 this is why i speculated if BD discs with rco's are suported in PS3
The list of documented PS3 disc formats is here, none of them uses rco's as far i know http://www.psdevwiki.com/ps3/PARAM.SFO#B...e_details:

(14/04/2015 04:43 AM)ZiNgA BuRgA Wrote: I'd prefer to explicitly state that it's an unknown ID though, otherwise it could get confusing with how some other numbers are represented in the XML (floats are numbers whilst integers are prefixed with 0x).

Good enought :)
I just imagined it could be usefull to have a list of the rco versions and i wrote this to take notes http://www.psdevwiki.com/ps3/Talk:Resour...O_versions

Has been written a bit on a rush, im sure there are some mistakes in the explain and some innacuracies but it will be rewritten at some point or removed in case is not needed anymore

Ive used the concept of "rco set" because it seems all the rco's from a specific firmware shares the same version, and suggested this version is refered to the "rco tool version" used to create the "rco set"
If you don't agree im fine to modify it, or you can modify yourself editing the page

The list of versions for PSP is complete ?... can be interesting to compare the rco versions of the last psp firmware... with the rco versions of the ps3 firmare published at this date

ZiNgA BuRgA · 19/04/2015 04:21 AM

(17/04/2015 09:36 AM)sandungas Wrote: What im going to say is speculative, but i think sony prepared some rare formats that later was not used, it seems there is a "BD music" and "BD photo"... and well...

BD-Audio is a standard format, I don't know about photos, though they can easily be emulated with the Bluray menu system.

(17/04/2015 09:36 AM)sandungas Wrote: because they could easilly implement support in ps3 to continue with what they was doing in psp using umd's with rco's
I always thought there are some undocumented disc formats for PS3 this is why i speculated if BD discs with rco's are suported in PS3

BD menus are completely different to how UMDs work (Java based system), so no, you won't find any RCOs there.
The design decisions for UMDs and BDs are completely different, so one shouldn't expect many similarities (beyond playing videos). For one, BDs are completely incompatible with what the PSP can do, ignoring the different physical format.

PS3 games may be a different thing, though until someone has seen it, I'd say it's unlikely too.

(17/04/2015 09:36 AM)sandungas Wrote: I just imagined it could be usefull to have a list of the rco versions and i wrote this to take notes http://www.psdevwiki.com/ps3/Talk:Resour...O_versions

Thanks for putting up that list.
I'm not sure how useful it will be though. I've never seen the value have much effect.

(17/04/2015 09:36 AM)sandungas Wrote: The list of versions for PSP is complete ?... can be interesting to compare the rco versions of the last psp firmware... with the rco versions of the ps3 firmare published at this date

I don't have a dump of FW6.61 handy so I can't check easily, but I wouldn't be surprised if it hasn't changed.

sandungas · 25/04/2015 05:04 PM

Ok, this talk now is taking us to the vsmx format, i didnt wanted to suggest big changes to don't request too much things to you (like the one i made in the previous message about parsing the gim settings from rcomage to gimconv)
My purpose was more to advise you of some small details, and i think i tolk about all in my previous messages

The next things are related to vsmx format, and the decoding/decompiling but some can be considered big changes that needs some work, and im not so sure if you are interested in vsmx after i realized only exists inside rco's from PSP UMD's
As far i know there are no vsmx scripts inside PS3 rco's... but are used inside PS3 coldboot.raf (the animation when the PS3 boots), and inside dynamic themes .p3t (the animation displayed at background of the theme)

The official tool to compile the vsmx scripts is named raf_script.exe is part of the "raf_compiler" tools from the sdk and is included in all the theme tools able to compile dynamic PS3 themes
Raf tools uses the fileformats .js (the plain "playstation javascript" code) and .jsx (the same file after encoding in vsmx format)
The .jsx is created automatically as an intermediary step of the process to compile a .raf file in a folder named "tmp"... and then raf_compiler.exe creates the .raf file with the .jsx inside
The easyer way to compile a .js thought... is simply by drag-and-drop the .js file over raf_script.exe, this generates a .jsx file with the same name

This is useful to know because is posible to use the classic method of test-error to compile multiple .js scripts until you have an exact match with the original .jsx (this way you can be 100% sure the .js is perfect)

----
I used the output of rcomage vsmxdec (and --decompile) as a start point to decompile PS3 coldboot.js
The first problem is PS3 scripts are VSMX format v2.0 (PSP only supports 1.0, right ?)
Rcomage crashes with PS3 scripts... because the vsmx format 2.0 has some additional opcodes specific for PS3

So i took your files and tryed to complete the missing "playstation javascript" code and errors in my coldboot.js by hand using the test-error method explained before with raf_script.exe and making lot of bynary comparisons with the original coldboot.jsx to see where was located the errors
At the same time i imagined this method could work (it took me some time to imagine and a week of tests) a friend named ada realized the official names of ALL the opcodes is inside the raf tools
So first wee made a table in wiki with all the official opcodes names and all the opcode names you used in rcomage to compare them and make some sense of all (also by doing this wee realized the new opcodes added in vsmx format v2.0 specific for PS3 are located at bottom of the table)
The opcode table is here, you can replace the opcode names in rcomage by the officials :)
http://www.psdevwiki.com/ps3/VSMX#OPCODE_Table

The full decoding (with official opcode names) and decompiling (to "playstation javascript" format) of PS3 coldboot.jsx i made is in the same page at bottom
What i did was to make a list of all the opcodes used, and the decoding line by line, all together (because is much better to have all the info in the same place for tests)
http://www.psdevwiki.com/ps3/VSMX#VSMX_D...ion_sample

Some weeks later, another friend named TizzyT (the same that helped me to find the last javascript statements in my adventure with the manual decompiling) decided to make a tool to decode and decompile the vsmx files named "jsxviewer"
I suggested him to send some changes requests to your git but he is a vs coder and prefered to start from scratch in vs to have more freedom, you can download jsxviewer from this link (is still in beta because it has some missing opcodes)
http://playstationhax.it/forums/topic/1060-jsxviewer/

I helped him to make the tool, suggestions, betatest it, etc... and it uses a format similar of what i was doing displaying all the info about decoding and decompiling all together in a .txt
This .zip contains an example of:
The original PS3 coldboot.jsx script (extracted raw from coldboot.raf)
-The decompiled .js (ready to compile again to .jsx, it should result in a file exactly like the original .jsx). Created by jsxviewer.exe
-A .txt file with all the decoding + decompiling info all together. Created by jsxviewer.exe
https://www.sendspace.com/file/wzlo2z

-----
This is mostly a resume about things related with vsmx, if you are interested in it wee can talk about wich opcodes are critical for PS3 support
If you try to decompile the coldboot.jsx in the .zip with rcomage you will realize there are some opcodes easy to fix that are the responsibles of the crashes in rcomage when trying to decompile it (specially what sony calls "vectors" that are simply arrays using the characters < and >)

ZiNgA BuRgA · 26/04/2015 03:37 AM

(25/04/2015 05:04 PM)sandungas Wrote: The first problem is PS3 scripts are VSMX format v2.0 (PSP only supports 1.0, right ?)
Rcomage crashes with PS3 scripts... because the vsmx format 2.0 has some additional opcodes specific for PS3

I've only ever seen the RAF compiler generate v2 scripts.
The v1.12 dev version posted earlier should at least be able to recognise v2 scripts, and has most of the opcodes (just not named the same as the official assembler).

Thanks for the nice table by the way :)

(25/04/2015 05:04 PM)sandungas Wrote: The full decoding (with official opcode names) and decompiling (to "playstation javascript" format) of PS3 coldboot.jsx i made is in the same page at bottom
What i did was to make a list of all the opcodes used, and the decoding line by line, all together (because is much better to have all the info in the same place for tests)
http://www.psdevwiki.com/ps3/VSMX#VSMX_D...ion_sample

Nice!

(25/04/2015 05:04 PM)sandungas Wrote: Some weeks later, another friend named TizzyT (the same that helped me to find the last javascript statements in my adventure with the manual decompiling) decided to make a tool to decode and decompile the vsmx files named "jsxviewer"
I suggested him to send some changes requests to your git but he is a vs coder and prefered to start from scratch in vs to have more freedom, you can download jsxviewer from this link (is still in beta because it has some missing opcodes)
http://playstationhax.it/forums/topic/1060-jsxviewer/

Haven't had the chance to test it yet, but looks like a pretty good tool that'll probably be better than Rcomage's ever will.
I'll probably complete the opcodes and make it compatible with the official tool, but I probably won't work on the decompiler, as it's relatively tricky to get correct.
Just use JSXViewer for all your needs :)

sandungas · 26/04/2015 11:09 AM

Thanks for all, i think are enought (maybe too much) suggestions and brainstormings by now, i filled 2 pages of the thread and i don't want to abuse of your hospitality, if there is some news about all wee was talking or some important change in psdevwiki that could benefit rcomage (for documenting/historic purposes) i will advise you guys

Do you plan to release a stable (not dev) rcomage v1.1.2 in the next weeks or months ? (no hurry, but i would like to know wich one should be considered the newest)

ZiNgA BuRgA · 26/04/2015 07:19 PM

I have very little free time these days so can't really make much progress on Rcomage.
The absolute latest will always be in the 'zinga' branch from GitHub, but you'll need to compile the code yourself.
'Stable' versions is largely arbitrary, and Rcomage is never really stable anyway =P

Don't worry about "abuse of hospitality" - ask whatever you want :) I'm happy to provide info anyway.
Whether or not I implement changes is another matter though.

Threaded Mode \| Linear Mode [PSP] Rcomage v1.1.1 - new RCO manipulation tool
Author	Message