The Kuwashii Challenge: .MOD script manipulation

The Kuwashii visor has been a sore sight to many since it's introduction in Mass Effect 2 due to the position of it's rotating reticule in relation to the eye. There have been several modders in the past that have released reticule position fixes for the visor for both ME2 and ME3, which was unfortunately only possible at the expense of it's dynamic reticule. This is because we as modders cannot influence the position of the animated texture overlay and are left with no option but to "clear" it and to paint a static reticule on the main visor texture instead.

I wanted to release my own visor/reticule fix combined with a re-textured framework to fit with my armor-sets, unfortunately I ran into an additional problem as it seems the dynamic reticule is shared with Garrus' visor which causes the reticule fix to indiscriminately break the dynamic reticule for both Shep and Garrus alike. Texmod and ME3Explorer use the hex identifiers for images to automatically match it's replacement across the board. For Texmod this is something that can't be manipulated and as such the situation for Garrus can only be resolved by sorting a static reticule directly onto his visor texture.

We do have some options using ME3Explorer while creating a texture replacement .MOD as we can manipulate the automatically generated mod script to skip certain PCC file references and only apply our modded textures to a selection of our choosing. Because this is (not yet) an automated process this has to be done by hand by going into an automatically generated script and adjusting the PCC references and IDs belonging to the texture instances.

You would assume that if Garrus and Shepard are both affected by a .MOD, the fix would be as simple as adjusting the replacement script for the reticule to only apply to PCC files associated with Shepard. 
To do that you'll first have to create a .MOD, what I did here was feed the TPF straight into Texplorer's mod generator and let it create a .MOD.

When you do this Texplorer will automatically match all instances of a texture and create a list of PCC files and IDs to replace in the script for every texture.

You'll notice that when I selected the dynamic reticule (TUR_HED_PROGarrus_Holo_Diff) in the list in ModGenerator that the info tab shows all PCC files and associates IDs it will add to the script.
I can't change that list from here but I can load in a generated .MOD through ModMaker and preview the script for the texture replacement jobs from there to edit them.

 As you can see, the script lists 8 PCC files total and the highlighted 2 are clearly associated with Shepard. I removed the 1st and 4th through 8th line here, only keeping the 2nd and 3rd.
Scrolling down a little I did the same on the ID's. Similarly there are 8 ID's listed, so I'm keeping the 2nd and 3rd and removing all others. The script repeats this list a bit further down, so I applied the same changes to that section. After editing the lists looked like this:

After having edited the script, I hit save changes on the preview windows and re-saved the .MOD so it would include the changes I made to the script.

In a perfect world the issue would now be resolved, unfortunately it showed through ingame testing that out of the 6 PCC files associated with Garrus, 4 use a material reference that ties in with Shepard. This meant that in practice, the script manipulation only worked for Garrus on the Normandy and the dynamic reticule was preserved there, but it stayed broken in Explore mode. As such I figured I would add a texture replacement job for a version of his visor including a static reticule like in the texmod version, but only for the instances in which his dynamic reticule was broken.

The method for this change is exactly the same as the one detailed above, with the exception of me keeping different PCC files in the list this time around. The edited script for Garrus' visor (Visor_Frame) ended up looking like this.

 Et voila!

Script manipulation is not the only tool available to us when trying to achieve a certain effect.
You can adjust meshes through a 3D editor, adjust the UV map (texture wrap), manipulate the texture through a photo/image editor like Photoshop or change the material assignment through Meshplorer. Which method serves your specific problem best often comes down to trial and error. In this case, after having tried all other options available to me, I settled on script manipulation as the best solution. I hope this will prove helpful to some.

The completed Texmod TPF and ME3Explorer .MOD for the Kuwashii Visor are featured in my BSN Thread.

Guide: ME3Explorer first time setup information

What is ME3Explorer?

ME3 Explorer is a fan-built toolset used to mod the Mass Effect Trilogy. Several features are still under development and new features are still actively being introduced. With it you can decompress, load, edit and save PCC files. You can mesh and texture mod and create content mods for ME3 within the constraints of the tools and the limitations of the game itself. The toolset is useable for mod creators and for those that simply want to apply already created mods.

Why this guide?

This guide is specifically meant for people looking to start using ME3Explorer as a means of permanently replacing textures and meshes in Mass Effect 3. I might add information specific to the tools for ME1 and ME2 as well at some point but this guide is not specifically tailored towards modding the older games.

First things first

Backup your vanilla install files in the Mass Effect 3 folder. The tool does not affect game saves or other Mass Effect 3 files except the ones listed below:

1. Make a backup of "PCConsoleTOC.bin" and "PCConsoleTOC.txt" in "..\Origin Games\Mass Effect 3\BioGame". You can rar/zip them up together and mark them as your vanilla backup.
2. Make a backup of your entire "CookedPCConsole" folder (all PCC and TFC files) in "..\Origin Games\Mass Effect 3\BIOGame\CookedPCConsole". Mark it as your vanilla backup. 

Which version should I install?

You can choose to download and install ME3Explorer one of two ways.

1. You can use the latest SVN release. It is used by the coders behind the project. Releases are referenced by their revision (REV) number. SVN is a Revision Control System: a repository of files, often the files for the source code of computer programs, with monitored access. Every change made to the source is tracked, along with who made the change, why they made it, references to problems fixed or enhancements introduced by the change. You can decide to use this version to be completely up to date and to help test bugs in the newest revisions of the program.
2. You can use the latest stable release from Sourceforge, which is often shorthanded to the "SF release". The SF is the latest SVN REV the creators deemed stable enough to transfer over. It will offer a choice between a Source and Binary release, you'll want to grab the latter. This version is specifically meant for those looking for the most stable solution that are not interested in bug-testing and helping development. It is always going to be slightly outdated. While most stable, this version is not necessarily bug-free.

As of this writing (updated Juli 29th, 2014) the most recent SF release is Version 0109k (Jul 7th, 2014). This the counterpart of SVN REV 653. The most recent SVN REV is currently 657.
Seeing this information is transient, it is always advised to check the bug report section on the ME3Explorer boards to make sure there are no mayor bugs with recent releases reported. It is also always good to simply ask if you are unsure how to proceed.

I normally always advise using the SVN as you will not be able to receive any support using older versions of ME3Explorer due to it being in constant development.At the moment however, my advice is to use the latest SF or SVN rev 653.

 ME3Explorer on Sourceforge           Tutorial on using SVN                    

ME3Explorer Bug report section       ME3Explorer version release News

Installation and first time setup

When downloading the SF you can place the Binary folder anywhere you find convenient (the SVN download is explained in the tutorial I linked above).
Now you'll want to navigate to the following locations to start up "ME3Explorer.exe": "..\Binary\" (SF) or "..\ME3Explorer\bin\Debug\" (SVN). 

ME3Explorer doesn't conventionally install so when you doubleclick the exe, the program will simply open. It is not yet ready for use though, as it needs to go through a one time setup procedure that will allow it to create and scan some necessary files in your game's install folder.

To start that process we'll have to start up Texplorer which is located under "Developer Tools".

You'll get a popup asking you if you want to edit DLC content as well.

This prompt allows you to pick whether you would like to mod the basegame only or the DLC content as well. While you might think there's no harm to include the DLC, there are several considerations before deciding which way to go with this so please carefully read the following section.

Why do we get this option at all?

Due to Origin's frantic DLC checking, it is impossible to mod DLC content without it throwing authentication errors once it's checked. As such, when you mod DLC content, you are forced to use a  'fix' that spoofs the DLC check communication for you and validates your DLC.
Download the DLC fix here: ME3 DLC Patcher. Simply copy the two provided files into "..\Origin Games\Mass Effect 3\Binaries\Win32\" and say yes when prompted to overwrite.

Scan without DLC

Texplorer will create a tree-structure file (me3tree.bin) of about 6mb in "..\ME3Explorer\bin\Debug\exec\" (SVN) or "..\Binary\exec\" (SF) that it can (re)use to visualize the game's file-structure and to enable you to browse through and manually replace texture files. Upon running your first mod Texplorer will additionally create a file called “CustTextures0.tfc" in "..\Origin Games\Mass Effect 3\BIOGame\CookedPCConsole" and it will create an entry for that file in your "PCConsoleTOC.bin" that tracks it’s size. The scan will take about 15 minutes to finish depending on your setup. 

Scan with DLC

Texplorer will create a treestructure file (me3tree.bin) of around 20mb in "..\ME3Explorer\bin\Debug\exec\" (SVN) or "..\Binary\exec\" (SF) that it can (re)use to visualize the game's file-structure and to enable you to browse through and manually replace texture files. It will scan the basegame and will unpack parts of the DLC SFAR contents in theie respective DLC folders. These files are necessary for DLC modding so you can't delete them once Texplorer is done generating the treestructure file. Upon running your first mod Texplorer will additionally create a file called “CustTextures0.tfc" in "..\Origin Games\Mass Effect 3\BIOGame\CookedPCConsole" and it will create an entry for that file in your "PCConsoleTOC.bin" that tracks it’s size. The scan can easily take up to 2 hours to finish depending on your setup. 

Note: Seeing DLC is unpacked during a scan it's a good idea to back up your unmodded DLC folder, though optional as you should create a backup of the unpacked state as well after treescan has completed.

After the scan

Texplorer generated your treestructure file called "ME3Tree.bin" in "..\ME3Explorer\bin\Debug\exec\" (SVN) or "..\Binary\exec\" (SF). To avoid unnecessary rescanning when something goes wrong for whatever reason, make a backup of your treestruct. You can mark it basegame or DLC, whichever is appropriate, and be sure to add the revision number it was created with as well.
You can do this by hand, or use Texplorer's tree export functionality introduced in SVN REV 628.

If you opted to scan with DLC, make a backup of your entire DLC folder so you preserve the vanilla post-scan state of the folders to again avoid unnecessary rescanning.
When scanning with REV 623+,  make a backup of your entire DLC folder after treescan is completed as it's in unpacked state after so to vanilla compatible with your new tree you'll need a backup.

Formats Explained 

.PCC - The PCC format is what Mass Effect uses to package the game's content. As PCC files are the entire package, they can overwrite any changes made to the same file before replacement, AKA PCC mods should always be applied first as .MODs and TPFs can do specific replacements inside these modded PCC files keeping previous changes intact as long as they don't overwrite the same object inside them.

.SFAR - The SFAR format is what Mass Effect uses to package the game's DLC content. Every individual DLC has it's own default.sfar located in uniquely named folders at "..\Origin Games\Mass Effect 3\BIOGame\DLC\".

.MOD - These are packages created by ME3Explorer that can hold binary, object and PCC replacement jobs. Because scripts used for the replacement jobs are included in the .MODs these are revision sensitive while the toolset is still under development, pay attention to which revision a .MOD was created with.

.TPF - Texmod is a utility to browse, extract, save and modify textures in Direct3D 9 applications. In order to share and distribute texture modifications, TPFs (Texmod Package Files) can be created which are compressed to prevent further modification. This modding method allows you to temporarily replace any texture in your application without the need to use an application specific modding tool or replacing any files in your application's installation. Using ME3Explorer, you can convert Texmod TPF's to the ME3Explorer .MODs format or you can load them in directly using the 'TPF/DDS Tools 2.0' and apply them to your game permanently.

.mod Compatibility

Be aware that DLC mesh .mods and basegame/DLC texture .mods created between REV592 and REV622 will no longer inherently work on REV623. This means the current SVN (628) is not compatible with the latest SF (600). TPFs still work as before through TPF/DDS Tools. In simple therms: Make absolutely sure .mods you are downloading are documented to be compatible with your version of ME3Explorer, be it SVN or SF release. Older mods can cause errors and/or simply won't apply correctly or at all.

You can use .mod fix capability in ModMaker post REV 628 to automatically fix the script for that .mod to work with the newer REVs. While this might work, we can't guarantee it will and best practice is always to seek out mods that are inherently compatible with the revision you are using.
We do recognize this isn't always possible.

To avoid issues with texture .mods caused by REV changes, you can opt to download .tpf files instead. The TPF/DDS Tools included in the toolset allow you to either directly apply a .tpf to the gamefiles as you would a .mod, or you can turn your .tpf into a .mod for your current revision.

Selecting mods

Any .MOD that was created with a DLC struct can be used to apply to basegame content, but whether it'll replace DLC content depends on your personal setup in regards to the treestruct you are using. Any .MOD that was created with a basegame struct is completely incapable of replacing DLC content, whether or not you are using a DLC struct yourself.

Sources for pcc / mod / tpf downloads:

Antarius' New Compilation of HR Tex (.MOD & fixed.tpf)	Varied texture mods
HR Textures by Ottemis	Varied texture & mesh mods
OTTE MODS	Hair mesh mods
ELE MODS	Varied texture & mesh mods
ELE's Tumblr	Hair mesh mods
Kani Hime's All Thumbs Production (PCC)	Varied texture & mesh mods
Voodooseason's Tumblr (PCC & .MOD)	Varied texture & mesh mods
Rana's Tumblr (PCC)	Hair mesh mods
wildeatheart27's Tumblr (PCC)	Hair mesh mods
nameislooney's Tumblr (PCC & UPK)	Hair mesh mods

Applying .mods

Applying .MODs is done using ModMaker, it's located under "Tools".

I personally always load through "Jobs", "Add Jobs to list" but you can also go through "File", "Load Jobs". To start applying your loaded mods, go to "Run", "Run all".

While applying mods ME3Explorer can run both hot in memory (but up to a shared cap of 4gigs) and hot on the CPU (up to 100% load on all cores), as such it can go unresponsive at times and lock up your system. You should be able to see it cycle through saving PCC files and applying the mod jobs in the debug window. Heavy mods are not just recognizable by the number of texture replacement jobs, they are also those in which a texture replacement job has a lot of instances in the game's PCC files and ends up having to use multiple threads to save each PCC file with. These are the mods that will spike your CPU usage and overburdening your system while these run is ill-advised. Should your program crash during mod installation, don't worry! Try re-applying the mods in question by loading in less at a time or go one by one and you should be able to reapply the mods without issues. You can also go through "Run", "Run Some" to check loaded jobs on and off. That way you can manually split an already created job list.

Applying .TPFs

In order to apply your Texmod TPF permanently to the game, the developers of ME3Explorer have created a nifty little thing called the "TPF/DDS Tools". It will allow you to load in any .tpf, check it's file formats for validity and if valid, apply it directly to the game as you would a .mod. You can also create your own fresh .mod for the textures with this tool.

This means that downloading a fixed format TPF will ensure you never have to pay attention to the Revision a texture-mod was built with in relation to the version you are using.

Launch ME3Explorer, go to "Tools" and pick "TPF/DDS Tools" in the dropdown menu.
In the window that pops up, click "Load TPF" and browse to the TPF you want to install.
Once you've opened it you'll see that the textures are listed now as is the texmod definition file (def). The definition file will help tell the game where these textures belong.

Every texture has it's own unique hex id or hash number (0xDB481023 for example), the game uses these to match them to the right textures in the game files. Texmod definition files provide the link between the filenames and the corresponding hex id's automatically when the tool can't find these id's in the filename itself. Now down at the bottom it'll tell you the Hashes were generated, click the "Analyse with Texplorer" button to let it scan if the formats are fitting for permanent replacement.
Once it's done it'll say "Finished Matching!". You can now hit "Install valid" to permanently install all valid textures to your game.

Now sometimes you might run into TPF's that are not in the right format and/or cannot be matched to their ingame counterparts. For these and other such issues I suggest you check out the more expansive tutorial on the TPF Tools by KFreon[/url] and the guide on fixing format issues with TPF's by KFreon.

Applying PCC mods

Some modders prefer to release mods by making a modded PCC file available for download rather than creating a .MOD file for mesh releases. ELE08 has written a nice guide on how to install PCC mods.
Be aware that PCC mods overwrite all other edits made to the PCC in question via regular mod application so always replace these first before you mod any other method. The same goes for content mod installers but they have their own install guides so I advise you seek them out and read them if you plan to use them.

Creating .MOD files from TPF

Texplorer was recently expanded with the ability to automatically generate a .MOD from it's base Texmod TPF as long as the file formats are correct.
This is usefull for all the TPF's that haven't been converted to .MODs yet, the recreation of old .MODs from their original TPF's and for people that experience issues with premade .MODs due to language version incompatibilities.
Unfortunately there are not a lot of active texture modders left and most released TPF's will have format issues for compatibility with ME3Explorer. Users like Antarius have made great contributions by creating fixed format TPF files with .MOD repacks by inactive modders to ensure people can easily use and repack their .MODs in the future. Here's a guide by KFreon on how to work with the TPF Tools.

Troubleshooting

When applying a .mod I am getting the following error:

Your version of ME3Explorer is newer than the .mod you are trying to apply and it is no longer compatible. Contact the .mod creator.

When applying a .mod, changes do not show up ingame, why is this?
If you are using REV623 or above, DLC mesh .mods created prior to that REV will no longer apply even though they don't throw a visible error. If this is your problem the debug window will show the following warning when cycling through the jobs it wants to apply:
"Error : Index was outside the bounds of the array."
If you encounter this, please alert the mod creator. Report any other issues to the bug report forums.


I'm getting a DLC authentication error after modding my game, how do I fix this?
As earlier mentioned in the guide, you will need to download a 'patch' that will enable you to mod DLC files while avoiding the authentication issue. Go here and download the files, copy the two provided files into "..\Origin Games\Mass Effect 3\Binaries\Win32\" and click yes when prompted to overwrite.

I'm getting a DLC corruption message after modding my game, how do I fix this?
When you apply mods to the gamefiles and they apply to DLC, they add to the overall size of the SFAR for the DLC in question. When these SFAR grow beyond 2gigs in size individually, they will start throwing corruption errors because the game can't process them properly anymore. To counter this you have two options.
1. Easy method, check which DLC have grown above 2gigs and vanilla them by copying your backup back.
2. Hard method, use the process described here: post8640.html#p8640

I vanilla'd my gamefiles through Origin update or repair but Texplorer still claims I have files modified after Juli 2013 and my treescan fails? 
Unless you throw out all modified files and redownload them from scratch or copy a backup back over that you know with 100% certainty was created right after a fresh install, just using update or repair via Origin will not work. If that is what you have done, you won't be able to see which files were originally modified because it seemingly resets the creation date back to where it should be. Still they are not kosher and you run risk of treescan falling over them. 
You can check the debug output for Texplorer to tell you which files treescan is complaining about and vanilla them properly to try again.

How do I vanilla my game and update my ME3Explorer installation so I can start fresh?
Vanilla basegame: Start by manually throwing away your "PCConsoleTOC.bin" and "PCConsoleTOC.txt" in "..\Origin Games\Mass Effect 3\BIOGame\", and any basegame file in your CookedPC folder that has a date newer than 4/25/2012. Alternately copy in a vanilla backup by hand, more viable for people with slower internet.
Vanilla DLC: Copy in the DLC backup you created AFTER you did your DLC treescan. If you never did a DLC treescan you don't need to do anything.

"Check for Update" by rightclicking ME3 in your Library in Origin, it will reodwnload any missing files. Now, throw out your entire ME3Explorer folder, you should already have a backup of you ME3Tree.bin! Download the latest SVN of ME3Explorer and put your old "ME3Tree.bin" back in the exec folder. When you launch Texplorer, it should tell you it found an existing tree and it won't trigger a rescan.

How do I vanilla my game to basegame and get rid of ME3Explorer completely?
If you have no interest in using ME3Explorer anymore and you want to vanilla, start by manually throwing away your "PCConsoleTOC.bin" and "PCConsoleTOC.txt" in "..\Origin Games\Mass Effect 3\BIOGame\" Now go into "..\Origin Games\Mass Effect 3\BIOGame\CookedPCConsole", sort by modified date and throw out any files that don't have a modified date of 4/25/2012. Be sure to include "CustTextures0.tfc". Now you'll want to "Check for Update" by rightclicking ME3 in your Library in Origin.

Alternately you can vanilla your PCC files by copying in a backup first, if your game is extensively modded and your internet connection speed is not amazing, this might be the better option for you as it won't have to redownload as many files. If you modded your DLC, you'll want to copy those over as well or manually undo changes by restoring the backups created by Texplorer. Method is described further down the troubleshoot section.

 
How do I avoid a rescan of the ME3Tree.bin in between updates?
You can at any given time update or completely refresh your version of ME3Explorer. To ensure that you don't have to unnessecarily rescan your Treestruct anytime you do, you'll want to hold on to a copy of your modified "PCConsoleTOC.bin" and the "me3tree.bin" and if you scanned DLC as well, a copy of your DLC folder. Remember that you cannot rescan the tree on a modded game, so this can save a lot of time.

I have copies of both treestructs, but switching doesn't seem to change anything.
This only applies since the REV 490 fix: Be sure to run Texplorer once between switching your treestruct and trying to apply .MODs to make sure the program is aware of the change.

I am trying to run a .MOD or manually replacing textures, why do I keep getting the following error?

When you first set up ME3Explorer and run your first mod, it will create "Custtextures0.tfc" and create an entry for that file in your "PCConsoleTOC.bin". Some mods seem to fail at generating this entry at times. If you followed this guide to the letter you will have a backed up copy of your modded TOC.bin which you can place back over the vanilla version to restore functionality.

How do I undo changes to basegame files made by a .MOD I just ran? 
You can most easily spot which PCC files were altered by going into your CookedPCConsole folder and sorting list by modified date. Any unaltered basegame files will have a modified date of 4-25-2012. Replace the PCC files with the appropriate modified date with the vanilla backup you have for them. Alternately, throw the modified files out manually and redownload through Origin.

How do I undo changes to DLC files made by a .MOD I just ran?
You can use the debug window to check which DLC files were edited while applying the .MOD:

In this example, DLC_HEN_PR was edited and as you can see, the program notes that an automatic backup was created while editing it. To revert changes all you have to do is go to "..\Origin Games\Mass Effect 3\BIOGame\DLC" and open DLC_HEN_PR, delete "Default.sfar" and rename "Default.bak" back to "Default.sfar". You can also copy your backup over the edited DLC folder.

ANY additional errors or problems should go directly to the official bugreport forums, NOT here.
This post is merely meant to catch most common issues to alleviate the pressure on the mods and admins on the ME3Explorer forums and to weed out known issues and newly discovered bugs that are yet to be resolved.


ME3Explorer bug report Forums

Now go forth and mod to your hearts content.