Thursday, October 29, 2015

Guide: DA:I Normals explained

Hello again for another episode of textures with Otte!

While I covered normals briefly in my "Guide: Texture information for aspiring modders", I opted to go into more detail now and really highlight the issues with the DA:I native normalmap format and how to work with them.

As I explained in the first guide, DA:I normals are in ATI2 (3Dc) format, it has two channels (X/Y instead of R/G/B). The DDS plugin decompresses the X/Y blocks and stores them in the R/G channels (the wrong way around).
I will be working with "DA3/Actors/BaseHeads/Inquisitor/Textures/hf_hed_inq_n" for this example.

As you can see, opening your file in Photoshop, Gimp and the Compressonator nets you flipped channel data! Now Photoshop has the added problem that, when the opened texture is flagged as a normal by the plugin, R/G/B is reconstructed from the R/G channels. As you can see, it messes up the normal. Here's how you fix it:

  • Open your extracted normalmap in Photoshop.
  • In the NVidia DDS Read Properties window select "Convert images to 32bits"
  • Upon opening, go to "Image → Mode → 16 bits/Channel". 
  • Select "Exposure and Gamma" from the dropdown and hit "OK".
  • Go to "Image → Adjustments → Levels..." and change the center value from 1.00 to 0.45
    < This workflow was taken from MediAsylum on DeviantArt, go call them a rockstar! >

Et voila, Photoshop falls in line with the rest:

Seeing the 3dc/ATI2 export flips the channels back correctly, if you'd just import and export your normals in your texture program without editing them, they'll come out right when you import them back into ModMaker. However, if you generate a new normalmap to overlay elements like you normally would without having flipped its channels, your details could end up the wrong way around ingame (extruded vs indented for ex).

To flip your channels in Photoshop:
  • Go to "Image → Adjustments →  Channel Mixer" and switch the red and green channel info (select red in the dropdown, change red 100% to 0 and green 0% to 100, then do the opposite for the green channel).
And in GIMP:
  • Go to "ColorsComponentsChannel Mixer" and switch the red and green channel info (select red in the dropdown, change red 100% to 0 and green 0% to 100, then do the opposite for the green channel). 

You can now generate and edit as you normally would. Before you save your edited normal, be sure to flip the channels back again.
Alternately you can also flip the channels on your generated normal before merging it with the original normal.

That's it for my normal advice. Hope you find it useful.

Thursday, January 1, 2015

DA:I - Male Elf sliderset "Deyvin"

Someone on Tumblr asked a question on how to create good looking male characters in DA:I.
My response was quite short, as it really just takes a bunch of messing around, no real 'trick' to it.
I figured I'd offer some premade characters people can straight up play as is - or use as a base for further customization. So, for those that don't like to spend ages in a CC and appreciate a sliderset over a save-game, preserving the ability to create your own with the Keep, this is for you!

No credit is necessary although obviously a link to the source would be appreciated if you are ever directly asked =) Redistribution prohibited, if I want to share this preset on other mod and/or sharing sites, I'll do this myself.

I plan to do more of these over time so keep an eye on my Tumblr or my blog here to stay up to date.

Some ingame shots of Deyvin:

And here are his slider settings:

And that's that, have any questions, shout, otherwise GAME ON.

Saturday, June 14, 2014

ME3Explorer: Using TPF/DDS Tools 2.0

In order to apply your Texmod TPF permanently to the game, the developers of ME3Explorer have created a nifty little thing called the "TPF/DDS Tools". It will allow you to load in any TPF, check it's file formats for validity and if valid, apply it directly to the game as you would a .mod. You can also create your own fresh .mod for the textures with this tool. This means that downloading a fixed format TPF will ensure you never have to pay attention to the Revision a texture-mod was built with in relation to the version you are using.

 Here's how you apply TPF's straight into the game:
Launch ME3Explorer, go to "Tools" and pick "TPF/DDS Tools" in the dropdown menu.

In the window that pops up, click "Load TPF" and browse to the TPF you want to install.

Once you've opened it you'll see that the textures are listed now as is the texmod definition file (def). The definition file will help tell the game where these textures belong.

Every texture has it's own unique hex id or hash number (0xDB481023 for example), the game uses these to match them to the right textures in the game files. Texmod definition files provide the link between the filenames and the corresponding hex id's automatically when the tool can't find these id's in the filename itself.
Now down at the bottom it'll tell you the Hashes were generated, click the "Analyse with Texplorer" button to let it scan if the formats are fitting for permanent replacement.

Once it's done it'll say "Finished Matching!". You can now hit "Install valid" to permanently install all valid textures to your game.


 Now sometimes you might run into TPF's that are not in the right format and/or cannot be matched to their ingame counterparts. For these and other such issues I suggest you check out the more expansive tutorial on the TPF Tools by KFreon and the guide on fixing format issues with TPF's by KFreon.

Monday, January 13, 2014

Guide: Texture information for aspiring modders


My background lies in traditional and digital art. This guide is meant for people like me, who don't have a background in game- or 3D design and won't inherently understand how things tie in together in this particular field. In retrospect, had I known what information to look for before beginning, I'd have had a less difficult time understanding how textures work and how they could be manipulated.

The Mass Effect series was created with an custom version of the Unreal3 Engine. The engine much determines how textures work and while there is some nuance between games, the general setup is largely the same. As an example, I've also done some modding for "Remember Me" and was easily able to with knowledge acquired modding the ME series.
I've experienced the same with "Dragon Age: Inquisition", which was built on the Frostbite 3 engine. So, while this guide was initially written for ME(3) texture modding, I've noticed that for DA:I specifically, the principle carries over with the exception of specmap material functions seeming more consistent and normal maps having a different format. The most important thing to remember is that most textures won't make sense until you view each channel separately and to always mirror against the channel usage of the original texture.

I've sat on this tutorial for a while, having it almost completed and it's sister sitting in my ME3 modding thread, with the rise of DA:I (texture) modding I figured it a good time to give a final edit and release it.

Texture maps
in ME3 and beyond

The following information might not bring anything new to some but it helps to start from the beginning so we can better understand what ME3 (or most any game for that matter) does with their textures and how things tie together. Working with textures for any game isn't so much about knowing exactly what they've done as it is about learning to recognize why and how they've done it. Lets take a look how Mass Effect works with texture maps and what you are likely to run into.

Remember that the way a texture functions is determined by the material and cannot be changed on the fly from the texture itself, as such it's important to look at the original closely when planning to edit a texture yourself.

Diffuse map

The diffuse is the most frequently used texture mapping method. It's a bitmap image wrapped directly onto the 3D mesh while displaying its original pixel color. This means that you can use any image as diffuse map to represent photo realistic quality. While some diffuse textures in Mass Effect will indeed hold colour information, others will be greyscale diffuse maps that get their colour information from a separate tintmap.

Function can be linked to all the different channels on a texture. Any single texture has up to 4 channels, R (red), G (green), B (blue) and, A (alpha). When you use the R/G/B channels together you can create a texture that holds it's own colour information. Most diffuse textures will also not have a functional alpha (or lack it entirely) and as such are most often a DXT1 without alpha to save on diskspace (file format). Those with alpha are DXT5. There are exceptions where colour is not put in the diffuse directly and the diffuse can be fit onto one channel rather than needing all three and that's where it can get confusing.

When creating a custom diffuse map I use both brushes and images for textures.
Good to remember is to avoid images that have lighting effect in them. While some highlighting and shading is done in the diffuse, most of it is controlled through the specmap and when you put it in the diffuse where it wasn't originally, it'll look all kinds of wrong.
On the flipside, if your diffuse does have lighting or shading in places put it back in! If it's in there, it's in there for a reason! I get this is hard for people working with a mouse but a lot of people don't do this and you end up with a flat diffuse that actually makes the whole look less realistic. This is also the reason I would suggest to anyone looking to seriously start texture modding to buy a drawing tablet. It doesn't have to be big, a bamboo is absolutely fine, it'll make this kind of work TONS easier. A big chunk of work I do on a diffuse will be manually painted on, having a tablet is a must for me.

If you are looking to introduce more detailing on a flat surface with little to no texture, you can use layer style screen or overlay to more easily add structure to it without having to recreate the entire surface. Be careful that you don't overuse this! There's really only a handful of places I've been able to use it where it worked, instead of making the surface look like a bleeding mess. It's not an auto-win button, but when it works, it saves time. Very situational.

Some tips:

  • Use 3DS! Load in your mesh and your diffuse texture so you can see what you are doing without having to take your texture ingame.
  • Group your materials together. Metal with metal, cloth with cloth. It'll make it easier to create the specmap later on.
  • Use masks to add colour to elements. This makes it easier to create recolours for your retexture later.
  • Put shading and highlighting on a separate layer and/or use masks for this as well.
  • Shading and highlighting, put it back on the diffuse when you retexture!
  • Learn how to use the pen tool, I can't express how important this tool is.
  • Make sure your patterned areas are the same size on each element. Element wrap sizes differ so having them all the same size on the texture does not mean they are the same size on the model, quite the opposite.
  • Take your time finding free stock (seamless) textures you can work with. You can make smart use of content aware fill in Photoshop (CS5+) to make a texture larger than it originally was. It's success depends on the textures structure, but be sure to look into it. The quality of a retexture halfly hangs on the quality of the stock used and smart texture selection.
Find common diffuse maps below to get an idea of what one might look like:

Specular map

You use specmaps to define the shininess and highlight colour of a surface. The higher the value of a pixel (from black to white), the shinier the surface will appear in-game. Therefore, surfaces such as dry stone or fabric would have a very dark specular map, while surfaces like metal or plastic would have lighter specular maps. 
The specular map is a peculiar thing in Mass Effect and has confused quite a lot of people in the past. Specmaps are inherently grey-scale, though in Mass Effect they won't look like they are.

While a diffuse needs the R/G/B channels you don't need those channels on a specmap. A DXT5 for example has 4 possible channels (R/G/B and alpha) and seeing the information held in a texture's channels is in essence just grey-scale information it's often so that these channels are independently used to map different functions. So, while Mass Effect and us modders call these textures the specmap, they are always a combination of several different maps spread over the channels of one texture which will include the specular map.
Putting all these different functions in channels of the same texture is essentially a filesize consideration. A texture with R/G/B and a functional alpha will be a DXT5 with interpolated alpha. Because this holds information for what would otherwise be 4 unique R/G/B channels (thus textures) and a DXT5 is about twice the size of a DXT1, they are saving half the diskspace (roughly put) while gaining flexibility.

Seeing a specular map is in essence grey-scale you'll sometimes see that the information on the R/G/B channels is identical resulting in a desaturated image. However, when you map things per channel the channels together make quite a colourful whole. Check out the specmap for this torso piece:

In this case the "typical" specmap most definitely sits on the green channel; as you can see it's a logical grey-scale with the fabric being slightly darker (less reflective) than the metal and carbon elements. The red and blue channels both completely black out certain elements which means these are both not a traditional specmap but are more likely to be glowmaps or maps that control sheen. The alpha channel clearly maps lights here.

While Mass Effect will often use the same colour channels for the same kind of functions, they're not completely consistent about their channels usage and that is really the hardest part of working with a specmap and creating a custom one. Finding out what function they've attached to each individual channel through the material itself and how to manipulate them to get the effect you desire. It's good to remember that, because any channel information is greyscale knowing that white is positive and black is negative, making a channel completely white will flood it's particular function across the surface the texture maps, alternately make the channel black to cancel out the effect it controls. It can be an effective way of finding out what any one channel effectively controls and how it makes elements look ingame.

Now the function attached to a specmaps channels isn't something you can manipulate via the texture itself, somewhere in the game (material) they hardcoded which channel maps which effect. This means that when you replace a mesh with another you might have to redo the specmap as the channels might not be hooked up to map and control the same functions.
Here are some of the functions that you can encounter mapped to one of the specmap's channels:
Glow, (coloured) Sheen, (coloured) lights, colour overlay, glossiness, transparency, material overlay effects (like glass for visors), animation frames (for instance animated text overlay cutoff points).
Basically every channel has it's own 'setting' in the material that determines exactly what kind of overlay effect it's tracking, all to make different material types like metal/leather and cloth look as realistic as possible in the engine.

Now sometimes you'll run into diffuse textures that don't seem to have a specmap to go with them, an example of this are the Mass Effect facial textures. Let's look at one of them:

The game needs a specmap to process an object in the lighting-engine, here it's actually hiding in the diffuse alpha. When an object doesn't need a lot of functions mapped to it, the number of channels/textures needed naturally drops where sometimes the separate specmap isn't necessary anymore. Most specmap textures will be DXT5 including an alpha, very rarely you'll encounter a DXT1 spec without alpha or a DXT1 with a 1 bit alpha.

Normal map
Normal Maps provide 3D detailing to an object. They are generated from the original high poly count 3D model of the mesh in question. As such, it's useful when creating a new normal map to work off the base of the original. Usually I upscale the original, depending on the object about half the size of the diffuse/spec unless it maps a full-body surface in which case I sometimes end up going up to 4096 on the diffuse and normal map alike. I then use the diffuse and Nvidia's normal map filter for Photoshop to generate a normal map.
I only tend to replace the original normal map in places where I changed the textures to such an extend as it being unavoidable. All other times I will smooth out the jagged edges on the original and overlay detailing generated with the Nvidia normal map preserving the original normal map.
ME3's normal maps are V8U8's. I save my edited normal maps as a BMP in Photoshop, then use the Direct X texture tool to convert to V8U8 and generate MIPs. Using a different file format will wreak havoc on your texture seams ingame. If you are modding a different game you can assume the same applies for it's native normalmap format and replacing it with one in a different format.

Some tips:
  • Using the Photoshop normalmap filter, dark elements are automatically indented and light elements extruded. If you are trying to generate a map for a metal surface with white scratches, on default settings the filter will extrude them instead of indent. Flip the Z axis if you want to reverse this. You can cut and paste bits together with the axis inverted or normal depending on what kind of material surface you are dealing with.
  • You can run the filter several times on different hardness (5-60) settings, then overlay each layer on a percentage to get a nice and fluid normal map with more depth.
  • If your normal map becomes too pitted and irregular due to the diffuse detailing, consider running a gaussian blur over the diffuse before you run the filter.
Find common normal maps below to get an idea of what one might look like:

I am adding a little note here in regards to DA:I normal maps as these have a different native format and will need to be edited differently. DA:I normals are in ATI2 (3Dc) format, it has two channels (X/Y instead of R/G/B). The Photoshop DDS plugin decompresses the X/Y blocks and stores them in the R/G channels (the wrong way around mind you). When the opened texture is flagged as a normal, R/G/B is reconstructed from the R/G channels. Unfortunately, this makes the normalmap look 'wrong' upon import into Photoshop.
Now I used to eyeball the changes necessary to convert the opened normal back to what it should be by manually switching the red and green channel and messing with the contrast for those channels. However, I've recently stumbled across this workflow by MediAsylum on DeviantArt which makes the process entirely foolproof. Go thank them for being awesome. Here's a step by step:
  • Open your extracted normalmap in Photoshop.
  • In the NVidia DDS Read Properties window select "Convert images to 32bits"
  • Upon opening, go to "Image - Mode - 16 bits/Channel". 
  • Select "Exposure and Gamma" from the dropdown and hit "OK".
  • Go to "Image - Adjustments - Levels..." and change the center value from 1.00 to 0.45
  • Go to "Image - Adjustments -  Channel Mixer" and switch the red and green channel info (select red in the dropdown, change red 100% to 0 and green 0% to 100, then do the opposite for the green channel).
Et voila. You can now edit your normals as you're used to. When you're happy with your changes, save it as a 3Dc through the DDS plugin.
For reference, below you'll see a normal as previewed in the tool, after export/opening in Photoshop, before flipping the channel info and after applying ALL changes as mentioned above. Note that an extracted normal opened in the Compressonator also has switched R/G channels (3rd image).
NB: If you previously got rid of the NVidia DDS Read Properties dialog popup (like I did), you can get it back by saving a random image as a DDS and pressing "Read Config..." in the save dialog window. This'll pop it back up and you can recheck the box to re-enable the dialog.

Tint map
Tintmaps are used to add colour to a surface with a greyscale diffuse. They function as a colour overlay on-top of the diffuse greyscale information (much like a colour overlay layer works in Photoshop) preserving all detail on the diffuse itself and making it possible to have multiple colours of an item without needing a custom diffuse for every variant.
In essence these are function maps, with information either positive (white) or negative (black).
Tintmap channels all mask function dictated by the material. As an example:

You can see how the R/G/B data together looks and how the channels separately look, if an alpha is present it would hold the same kind of black/white data as on the R/G/B, masking parts of the texture surface for material control. You can't manipulate the function each channel maps from the texture, this is determined by the material itself.

Stripe map 
Stripemaps are used to add pattern-colour to a 3D model with a greyscale diffuse and tintmap. They function as a colour overlay on-top of the diffuse greyscale information, much like the tintmap.

Wednesday, September 25, 2013

The Kuwashii Challenge: .MOD script manipulation

The Kuwashii visor has been a sore sight to many since it's introduction in Mass Effect 2 due to the position of it's rotating reticule in relation to the eye. There have been several modders in the past that have released reticule position fixes for the visor for both ME2 and ME3, which was unfortunately only possible at the expense of it's dynamic reticule. This is because we as modders cannot influence the position of the animated texture overlay and are left with no option but to "clear" it and to paint a static reticule on the main visor texture instead.

I wanted to release my own visor/reticule fix combined with a re-textured framework to fit with my armor-sets, unfortunately I ran into an additional problem as it seems the dynamic reticule is shared with Garrus' visor which causes the reticule fix to indiscriminately break the dynamic reticule for both Shep and Garrus alike. Texmod and ME3Explorer use the hex identifiers for images to automatically match it's replacement across the board. For Texmod this is something that can't be manipulated and as such the situation for Garrus can only be resolved by sorting a static reticule directly onto his visor texture.

We do have some options using ME3Explorer while creating a texture replacement .MOD as we can manipulate the automatically generated mod script to skip certain PCC file references and only apply our modded textures to a selection of our choosing. Because this is (not yet) an automated process this has to be done by hand by going into an automatically generated script and adjusting the PCC references and IDs belonging to the texture instances.

You would assume that if Garrus and Shepard are both affected by a .MOD, the fix would be as simple as adjusting the replacement script for the reticule to only apply to PCC files associated with Shepard. 
To do that you'll first have to create a .MOD, what I did here was feed the TPF straight into Texplorer's mod generator and let it create a .MOD.
When you do this Texplorer will automatically match all instances of a texture and create a list of PCC files and IDs to replace in the script for every texture.

You'll notice that when I selected the dynamic reticule (TUR_HED_PROGarrus_Holo_Diff) in the list in ModGenerator that the info tab shows all PCC files and associates IDs it will add to the script.
I can't change that list from here but I can load in a generated .MOD through ModMaker and preview the script for the texture replacement jobs from there to edit them.
 As you can see, the script lists 8 PCC files total and the highlighted 2 are clearly associated with Shepard. I removed the 1st and 4th through 8th line here, only keeping the 2nd and 3rd.
Scrolling down a little I did the same on the ID's. Similarly there are 8 ID's listed, so I'm keeping the 2nd and 3rd and removing all others. The script repeats this list a bit further down, so I applied the same changes to that section. After editing the lists looked like this:

After having edited the script, I hit save changes on the preview windows and re-saved the .MOD so it would include the changes I made to the script.

In a perfect world the issue would now be resolved, unfortunately it showed through ingame testing that out of the 6 PCC files associated with Garrus, 4 use a material reference that ties in with Shepard. This meant that in practice, the script manipulation only worked for Garrus on the Normandy and the dynamic reticule was preserved there, but it stayed broken in Explore mode. As such I figured I would add a texture replacement job for a version of his visor including a static reticule like in the texmod version, but only for the instances in which his dynamic reticule was broken.

The method for this change is exactly the same as the one detailed above, with the exception of me keeping different PCC files in the list this time around. The edited script for Garrus' visor (Visor_Frame) ended up looking like this.
 Et voila!

Script manipulation is not the only tool available to us when trying to achieve a certain effect.
You can adjust meshes through a 3D editor, adjust the UV map (texture wrap), manipulate the texture through a photo/image editor like Photoshop or change the material assignment through Meshplorer. Which method serves your specific problem best often comes down to trial and error. In this case, after having tried all other options available to me, I settled on script manipulation as the best solution. I hope this will prove helpful to some.

The completed Texmod TPF and ME3Explorer .MOD for the Kuwashii Visor are featured in my BSN Thread.