[TUTORIAL] Texture batch conversion and modification (Albedo-ready)

FuNK!

FuNK!

Registered
Hey there,
if you have ever tried to convert a track or mod with a massive amount of textures you will surely know that it takes a lot of time to edit the DDS textures one by one to suit the Albedo requirements by rFactor2.

Tutorial for ImageMagick and DirectXTex (by @FuNK!)
Note: Any of the commands below can just be copy-pasted. You don't need to re-type them!
Note: It was reported that on some systems you have to add "magick" before the "mogrify" command.

1. Install ImageMagick
Get the latest version here: ImageMagick for Windows

2. Download the DirectXTex tools
Get the latest version here: Microsoft DirectXTex releases on GitHub
You have to download »texconv.exe« and »texdiag.exe«.

3. (Optional) Download @Lazza's "rF2_TexFix" program
This tool replaces the manual sorting process as described in section 7, as well as the modification and conversion of the textures as described in section 8.
Simply place the rF2_TexFix.exe in your root directory (see section 5)
https://www.dropbox.com/s/qols3xqtnoqn4yd/rF2_TexFix.zip?dl=0

4. (Optional) Install the Cmder command-line tool
This step is optional because you can just use the Windows command shell (cmd). Cmder is for the power users (installation is manual, Cmder provides some more/better features).
Get the latest version here: Cmder releases on GitHub
Follow the installation instructions here.

5. Prepare folders and tools
Choose a directory where you want to do the texture conversions and edits. Create the following (sub-)directories there:
Code: 
\dds-source
\dds-source\dxt1
\dds-source\dxt5
\png-dxt1
\png-dxt5
\output
Now copy the DirectXTex tools »texconv.exe« and »texdiag.exe« into the root directory.

(Info) You may chose your own directory names, but then you have to modify the commands below to suit this.

6. Open a command shell in the root folder
Open a new Windows command shell (cmd) in the root folder by press and hold shift key and rightclick somewhere in a empty area. From the menu select »Open command window here«.

(Optional) If you use Cmder and have installed it properly, you can do a rightclick somewhere in the folder and select »Cmder here«.

7. Copy your source DDS files and sort them according to their format
(Info) You can skip this step, if you already know the DDS formats. Just move the files in the respective »dxt*« sub-directory of »dds-source«. Sorting is necessary because after conversion the source DDS to PNG files the DDS format information is lost and a conversion back to DDS will automatically convert the PNG to DXT5 DDS files.

(Info) Please exclude any cubemaps, bumpmaps or specmaps from the source files as the don't need to be modified.

Copy all your source DDS files in the »source« folder. Run the following command to gather the format information:
Code: 
texdiag info dds-source\*.dds > output.txt && START output.txt
This command creates an »output.txt« file in the root directory which includes all basic information of the DDS files in the »source« directory. The »output.txt« will be directly opened with the Editor (it's the Windows default, of course you may use another program). You can avoid this by only using the first part of this command (until »&&«).
Screen the »output.txt« for BC1 (a.k.a. DXT1) and BC3 (a.k.a. DXT5). Move the files into the respective sub-directory in your source directory (DXT1 to »dxt1« and »DXT5 to »dxt5«).

(Info) Normally the files in DXT1 format are the minority, so moving these first might be the fastest way to sort the files.

8. Convert, modify and convert back with ImageMagick and texconv
(Info) ImageMagick gives the best lossless results for PNG conversion, so this is the preferred method. Texconv can do the conversion too, but the results are compressed too much which will result in even worse DDS later. Therefore I don't include the alternative commands here, but if you are interested have a look into the DirectXTex Wiki.
Run the following command:
Code: 
mogrify -format png -define png:compression-level=0 -define png:compression-filter=5 -define png:compression-strategy=2 -depth 8 -channel RGB +level 7.8%,78.4% -path png-dxt1\ dds-source\dxt1\*.dds >> log.txt && mogrify -format png -define png:compression-level=0 -define png:compression-filter=5 -define png:compression-strategy=2 -depth 8 -channel RGB +level 7.8%,78.4% -path png-dxt5\ dds-source\dxt5\*.dds >> log.txt && texconv -nologo -timing -y -f DXT1 -o output png-dxt1\*.png >> log.txt && texconv -timing -nologo -y -f DXT5 -o output png-dxt5\*.png >> log.txt
This command will convert the DDS files to PNG based on their source folders (i.e. format) and modify the color levels at the same time. The last part of the command then converts the modified PNG back to DDS and save them in the »output« directory. During all steps the console output gets written into a log.txt file (that's much easier than scrolling the console)

Please note: In case you only have DXT1 or DXT5 textures the above command will fail. You need to use the following separated commands instead:

For DXT1:
Code: 
mogrify -format png -define png:compression-level=0 -define png:compression-filter=5 -define png:compression-strategy=2 -depth 8 -channel RGB +level 7.8%,86.3% -path png-dxt1\ dds-source\dxt1\*.dds >> %FileDateTime%_log.txt && texconv -nologo -timing -y -f DXT1 -o output png-dxt1\*.png >> %FileDateTime%_log.txt

For DXT5:
Code: 
mogrify -format png -define png:compression-level=0 -define png:compression-filter=5 -define png:compression-strategy=2 -depth 8 -channel RGB +level 7.8%,86.3% -path png-dxt5\ dds-source\dxt5\*.dds >> %FileDateTime%_log.txt && texconv -nologo -timing -y -f DXT5 -o output png-dxt5\*.png >> %FileDateTime%_log.txt

Now you are done and that just took (without the installation of the tools) just a couple of minutes instead of hours or days! Awesome, right?

Tutorial for Photoshop (by @KittX)
(Info) This tutorial uses the macro-capabilities of Photoshop. Follow these simple steps to configure your macro to batch-edit DDS files. You have to record one macro for DXT1 files and one for DXT5. You can follow the steps 4 to 6 from the tutorial above to sort your files.

1. Open one of the files to edit in Photoshop, go to Window-Actions, and you have actions tab near the history tab.
2. Press "Create new action" (sheet icon), name it appropriately
3. Press record. Now every action you make is recorded. But it can be then removed and re-recorded if needed.
4. Apply color levels , and save it in the needed dds format*. It will record the dds saving settings, so you can make 2 sets of actions - for dxt1 and dxt5 for example.
5. Once you've saved, press stop and your action is ready to use.
6. Go to File > Automate > Batch, choose your action, choose folder and it'll convert your files.
*But make sure you revert the changes to the file you've opened for recording actions, and save it before going batch process, or make sure it is in another folder. Otherwise it will open that file again, and will apply levels again.

========================
Please note: DDS is a compressed format. Open a DDS, edit it and save it again will unfortunately result in a loss of quality. Make sure to not do more than maybe one or too editing steps. Better solution is to go with original files (e.g. with skins or textures in their PSD format).

Let me know your comments and report any problems/difficulties. And maybe someone is willing to improve this to a full tool (batch file, etc.)!? All programs are open source, so even redistribution in your own program would be allowed ;)

Credits:
Thanks @Marcel Offermans for (accidentally) reminding me of ImageMagick's great features and possibilities!
Thanks to @KittX to add how you could achieve batch-editing with Photoshop!
Thanks to @Lazza for providing the little program to translate the manual process into a automatic one!!
 
Last edited:
Thank you very much @FuNK!
This is very useful
Is it possible that there is an error in this command?
Code: 
texdiag info source\*.dds > output.txt && START output.txt
Is it possible that it is
Code: 
texdiag info dds-source\*.dds > output.txt && START output.txt
 
lagg said:
Thank you very much @FuNK!
This is very useful
Is it possible that there is an error in this command?
Code: 
texdiag info source\*.dds > output.txt && START output.txt
Is it possible that it is
Code: 
texdiag info dds-source\*.dds > output.txt && START output.txt
Click to expand...
Good eyes! Thanks, I fixed it ;)
 
Thanks for your work and your tuto @FuNK!

What about BC2 and B8G8R8A8 Format ?

When i run the following command "mogrify -format png -define png:compression-level ...." ... the CMD return "unknown command ...."
 
Last edited:
I know it's maybe a stupid question but did you install ImageMagick properly?
If so, please try the first part of the command only (until the &&). If you still get that error please add the command “magick“ before mogrify.
 
Ok Thanks ... I have made a mistake. It work now.

When I run the command ... it was in the wrong folder :p
 
Last edited:
FuNK! said:
Run the following command:
Code: 
mogrify -format png -define png:compression-level=0 -define png:compression-filter=5 -define png:compression-strategy=2 -depth 8 -channel RGB +level 7.8%,78.4% -path png-dxt1\ dds-source\dxt1\*.dds >> log.txt && mogrify -format png -define png:compression-level=0 -define png:compression-filter=5 -define png:compression-strategy=2 -depth 8 -channel RGB +level 7.8%,78.4% -path png-dxt5\ dds-source\dxt5\*.dds >> log.txt && texconv -nologo -timing -y -f DXT1 -o output png-dxt1\*.png >> log.txt && texconv -timing -nologo -y -f DXT5 -o output png-dxt5\*.png >> log.txt
Click to expand...
Where (in what folder) should I run this command?
 
There's another way, if you're using Photoshop you can record the actions and then do batch convert within the program, so no installing other tools required.
 
KittX said:
There's another way, if you're using Photoshop you can record the actions and then do batch convert within the program, so no installing other tools required.
Click to expand...
Of course you can, but don't you need Photoshop for that? :D
If you like, this tut helps those who don't have PS or don't want to spend money on it (like I do). Anyway, if you would like to write a short tut for the PS way, I would still welcome it (more options is allways a good thing) ;)
 
Yup, my post was just to make sure Photoshop people are aware that it can be done within the prorgam.
Here ya go:

• open one of the files to edit, go to Window-Actions, and you have actions tab near the history tab.
• Press "Create new action" (sheet icon), name it appropriately, and
• Press record. Now every action you make is recorded. But it can be then removed and re-recorded if needed.
• Apply levels, and save it in the needed dds format. It will record the dds saving settings, so you can make 2 sets of actions - for dxt1 and dxt5 for example.
• Once you've saved, press stop and your action is ready to use.
• Go to file-automate-batch, choose your action, choose folder and it'll convert your files.

*But make sure you revert the changes to the file you've opened for recording actions, and save it before going batch process, or make sure it is in another folder. Otherwise it will open that file again, and will apply levels again.
 
Last edited:
I've done the same with PS (a version of it was free at one point - not sure if it still is) but this external tools method is attractive as it obviously doesn't need PS, plus it's more open to automation. Are there exotic formats to worry about? I've done some texture fixing (albedo) and needed to allow for DXT1, DXT3, and DXT5. Could there also be more?

It should be possible to automate the separating of the different formats, making the exclusion of spec and bump maps the most manual (annoying) part, but would need to be sure of the various formats first.
 
Lazza said:
I've done the same with PS (a version of it was free at one point - not sure if it still is) but this external tools method is attractive as it obviously doesn't need PS, plus it's more open to automation. Are there exotic formats to worry about? I've done some texture fixing (albedo) and needed to allow for DXT1, DXT3, and DXT5. Could there also be more?

It should be possible to automate the separating of the different formats, making the exclusion of spec and bump maps the most manual (annoying) part, but would need to be sure of the various formats first.
Click to expand...

You probably mean the CS2. I don't want to go off topic, so I only say that it wasn't and will never be free at all - it's a violation of copyright if you use it without having purchased the license. That's no option for me (and many others). Btw. with GIMP and it's Script-Fu you would be able to write macros too. This again is an argument pro free software again.

I believe it's possible to sort the files based on their format automatically based on the output of texdiag.exe. At least it generates an output so with some programming skills you should be able to send the textures one by one to the tool, read out if its DXT1, DXT3 or DXT5, and "progress is" (move to a folder or convert/modify it with ImageMagick and convert it back to DDS with Texconv.exe). Unfortunately I cannot do that because my skills aren't sufficient. I tried to write a command script (a.k.a. batch file *.BAT or *.CMD) but failed (e.g. the parameters for mogrify were not handed over properly).

Exotic formats. Yeah, that DXT3 is almost one. AFAIK it was recommended to use DXT5 instead (you can sort the DXT3 to the DXT5 when processing them). Other formats are Cubemaps and Normalmaps because of their different code. Furthermore there are some DDS with special features which wouldn't load properly in rF2 (e.g. YCoCg DXT5 or RXGB DXT5 if I remember correctly). So in case you see something unusual in the output from Texdiag.exe which is not equal to DXT1 or DXT5, you should have a look what file it is and check if it really needs the fixing or if it needs to be converted to either DXT1 or DXT5.
 
So I got around to trying this, it seems I need to run the command as "magick mogrify" rather than just "mogrify" - does this sound reasonable?

For my use I'm throwing together a quick .exe to automatically generate the texdiag file, sort the source images into their respective folders, and then convert-manipulate-reconvert them all to the output folder. But I'm having to use the "magick mogrify" command so curious if that's just me...

*Here's my .exe, it shortcuts the above process from Step 4; so just put it in a folder along with texdiag and texconv (and mogrify available), with a source\ folder with your DDS files in it, and it will do the rest. No need for extra folders, it'll create a working folder and an output folder.
https://www.dropbox.com/s/qols3xqtnoqn4yd/rF2_TexFix.zip?dl=0
 
Last edited:
Lazza said:
So I got around to trying this, it seems I need to run the command as "magick mogrify" rather than just "mogrify" - does this sound reasonable?

For my use I'm throwing together a quick .exe to automatically generate the texdiag file, sort the source images into their respective folders, and then convert-manipulate-reconvert them all to the output folder. But I'm having to use the "magick mogrify" command so curious if that's just me...

*Here's my .exe, it shortcuts the above process from Step 4; so just put it in a folder along with texdiag and texconv (and mogrify available), with a source\ folder with your DDS files in it, and it will do the rest. No need for extra folders, it'll create a working folder and an output folder.
https://www.dropbox.com/s/qols3xqtnoqn4yd/rF2_TexFix.zip?dl=0
Click to expand...

Yeah in some cases it seams you need to run the commands with magick. Normally all commands should be integrated in the system if you have just installed it with their setup files. However, I believe that some Firewall/Antivirus software may influence the normal behavior.

Anyway, thank you very much for your effort! Would you mind sharing the code too? Maybe that could help others if they decide to build a software for the whole process?

EDIT: I've added this tool into the first post + tutorial. Thanks!
 
Last edited:
@FuNK! Yeah, antivirus makes sense probably. I wrote my prog to try just 'mogrify' first, then try what I had to use as a backup option.

This is just a simple/ugly AutoIT script based on your commands.

Code: 
#include <FileConstants.au3>
#include <StringConstants.au3>
#include <Array.au3>
AutoItSetOption("MustDeclareVars", 1)
Local $comDiag = "texdiag"
Local $comConv = "texconv"
Local $comMogrify = "mogrify"
SplashTextOn("rF2_TexFix", "Gathering info...", Default, 100)
Sleep(2000)
RunWait(@ComSpec & " /c "&$comDiag&" >texinfo.txt")
If (FileGetSize("texinfo.txt") < 100) then
    SplashOff()
    MsgBox(4096 + 16, "rF2_TexFix", "texdiag not available!")
    Exit
endif
RunWait(@ComSpec & " /c "&$comConv&" >texinfo.txt")
If (FileGetSize("texinfo.txt") < 100) then
    SplashOff()
    MsgBox(4096 + 16, "rF2_TexFix", "texconv not available!")
    Exit
endif
RunWait(@ComSpec & " /c "&$comMogrify&" >texinfo.txt")
If (FileGetSize("texinfo.txt") < 100) then
    $comMogrify = "magick mogrify"
    RunWait(@ComSpec & " /c "&$comMogrify&" >texinfo.txt")
    If (FileGetSize("texinfo.txt") < 100) then
        SplashOff()
        MsgBox(4096 + 16, "rF2_TexFix", "mogrify not available!")
        Exit
    EndIf
endif

Local $sSourceFolder = "source"
Local $sWorkingFolder = "working"
Local $sOutputFolder = "output"
Local $sDirSep = "\"
RunWait(@ComSpec & " /c " & $comDiag &" info source\*.dds > texinfo.txt")
Local $hFileOpen = FileOpen("texinfo.txt", $FO_READ)
if $hFileOpen = -1 Then
    Exit
EndIf
DirCreate($sWorkingFolder)
DirCreate($sOutputFolder)
Local $sLine
Local $sCurFilename, $sCurFormat
Local $Abort = false
Local $asFormatList[1] ; array storing names of formats/folders. We're starting with size of 1 and will always have a 'blank' at the end
While (Not $Abort)
    $sline = FileReadLine($hFileOpen)
    if (@error <> 0) then
        $Abort = true
    Else
        $sline = StringStripWS($sline, $STR_STRIPLEADING + $STR_STRIPTRAILING )
        if (StringLeft($sLine,StringLen($sSourceFolder)+1) = $sSourceFolder & $sDirSep AND StringRight($sLine,4) = ".dds") Then
            $sCurFilename = StringMid($sLine,StringLen($sSourceFolder)+2)
        EndIf
        if (StringLeft($sLine,9) = "format = ") Then
            $sCurFormat = StringMid($sline, 10)
            if (_ArraySearch($asFormatList, $sCurFormat) = -1) Then ; haven't had this format before, let's add it
                ReDim $asFormatList[UBound($asFormatList) + 1]
                $asFormatList[UBound($asFormatList)-2] = $sCurFormat
                DirCreate($sWorkingFolder & $sDirSep & $sCurFormat)
            EndIf
            FileCopy($sSourceFolder & $sDirSep & $sCurFilename, $sWorkingFolder & $sDirSep & $sCurFormat)
        EndIf
    EndIf
WEnd
FileClose($hFileOpen)
; we now have all the source files in their subfolders in our working folder, and an array with the list of formats - so now we can set up the command to fix and convert them all
Local $sCommand = @ComSpec & " /c " ; set up the command
for $i = 0 to (UBound($asFormatList) - 2)
    ; now add the png conversion command for this format
    $sCommand &= $comMogrify&" -format png -define png:compression-level=0 -define png:compression-filter=5 -define png:compression-strategy=2 -depth 8 -channel RGB +level 7.8%,78.4% -path "&$sWorkingFolder&$sDirSep&$asFormatList[$i]&$sDirSep&" "&$sWorkingFolder&$sDirSep&$asFormatList[$i]&$sDirSep&"*.dds >> log.txt && "
    ; note we're adding && at the end of each command here
Next
for $i = 0 to (UBound($asFormatList) - 2)
    ; now to convert back to dds files in the output folder
    $sCommand &= $comConv&" -nologo -timing -y -f "&$asFormatList[$i]&" -o "&$sOutputFolder&" "&$sWorkingFolder&$sDirSep&$asFormatList[$i]&$sDirSep&"*.png >> log.txt"
    if $i < (UBound($asFormatList) - 2) Then
        $sCommand &= " && " ; not the final command, so get ready to add another next loop
    EndIf
Next
SplashTextOn("rF2_TexFix", "Running commands...", Default, 100)
; now we can run the command
RunWait($sCommand)
SplashOff()
Local $res = MsgBox(4096 + 4, Default, "Finished. Would you like to remove all temp files/folders?")
if $res = 6 Then
    ; user wants to remove all the temp stuff
    DirRemove($sWorkingFolder,1)
EndIf
 
You must log in or register to reply here.
Back
Top