Textures 纹理
Our rasterizer can render objects like cubes or spheres. But we usually don’t want to render abstract geometric objects like cubes and spheres; instead, we want to render real-world objects, like crates and planets or dice and marbles. In this chapter, we’ll look at how we can add visual detail to the surface of our objects by using textures.
我们的光栅器可以渲染立方体或球体等物体。但我们通常不想渲染立方体和球体这样的抽象几何物体;相反,我们想渲染真实世界的物体,比如板条箱和行星,或者骰子和弹珠。在这一章中,我们将看看如何通过使用纹理为我们的物体表面添加视觉细节。
Painting a Crate 绘画板条箱
Let’s say we want our scene to have a wooden crate. How do we turn a cube into a wooden crate? One option is to add a lot of triangles to replicate the grain of the wood, the heads of the nails, and so on. This would work, but it would add a lot of geometric complexity to the scene, resulting in a big performance hit.
假设我们想让我们的场景有一个木箱。我们如何将一个立方体变成一个木箱呢?一种选择是添加大量的三角形来复制木头的纹理、钉子的头等等。这也行得通,但它会给场景增加很多几何上的复杂性,导致性能上的巨大冲击。
Another option is to fake the details: instead of modifying the geometry of an object, we just “paint” something that looks like wood on top of it. Unless you’re looking at the crate from up close, you won’t notice the difference, and the computational cost is significantly lower than adding lots of geometric detail.
另一个选择是伪造细节:我们不修改物体的几何形状,而只是在它上面 "画 "一些看起来像木头的东西。除非你从近处看这个箱子,否则你不会注意到区别,而且计算成本明显低于添加大量几何细节。
Note that the two options aren’t incompatible: you can choose the right balance between adding geometry and painting on that geometry to achieve the image quality and performance you require. Since we know how to deal with geometry, we’ll explore the second option.
请注意,这两个选项并不是不相容的:你可以在添加几何体和在几何体上绘画之间选择适当的平衡,以达到你所要求的图像质量和性能。由于我们知道如何处理几何体,我们将探讨第二种选择。
First, we need an image to paint on our triangles; in this context, we call this image a texture. Figure 14-1 shows a wooden crate texture.
首先,我们需要一个图像来画在我们的三角形上;在这里,我们称这个图像为纹理。图14-1显示了一个木箱的纹理。
Next, we need to specify how this texture is applied to the model. We can define this mapping on a per-triangle basis, by specifying which points of the texture should go on each vertex of the triangle (Figure 14-2).
接下来,我们需要指定这个纹理如何应用到模型上。我们可以在每个三角形的基础上定义这种映射,指定纹理的哪些点应该放在三角形的每个顶点上(图14-2)。
To define this mapping, we need a coordinate system to refer to points in the texture. Remember, a texture is just an image, represented as a rectangular array of pixels. We could use and coordinates and talk about pixels in the texture, but we’re already using these names for the canvas. Therefore, we use and for the texture coordinates and we call the texture’s pixels texels (a contraction of texture elements).
为了定义这种映射,我们需要一个坐标系统来指代纹理中的点。请记住,纹理只是一个图像,表示为一个矩形的像素阵列。我们可以使用 x 和 y 坐标来谈论纹理中的像素,但我们已经在画布上使用了这些名称。因此,我们用u和v来表示纹理坐标,我们把纹理的像素称为texels(纹理元素的缩略语)。
We’ll fix the origin of this coordinate system at the top-left corner of the texture. We’ll also declare that and are real numbers in the range , regardless of the actual texel dimensions of the texture. This is very convenient for several reasons. For example, we may want to use a lower- or higher-resolution texture depending on how much RAM we have available; because we’re not tied to the actual pixel dimensions, we can change resolutions without having to modify the model itself. We can multiply and by the texture width and height respectively to get the actual texel indices and .
我们将这个(u,v)坐标系的原点固定在纹理的左上角。我们还将声明u和v是[0,1]范围内的实数,与纹理的实际文本尺寸无关。这有几个原因,非常方便。例如,我们可能想使用较低或较高的分辨率的纹理,这取决于我们有多少可用的RAM;因为我们不受实际像素尺寸的约束,我们可以改变分辨率而不必修改模型本身。我们可以用u和v分别乘以纹理的宽度和高度来得到实际的texel索引tx和ty。
The basic idea of texture mapping is simple: we compute the coordinates for each pixel of the triangle, fetch the appropriate texel from the texture, and paint the pixel with that color. But the model only specifies and coordinates for the three vertices of the triangle, and we need them for each pixel . . .
纹理映射的基本思想很简单:我们计算三角形每个像素的(u,v)坐标,从纹理中获取适当的texel,然后用该颜色涂抹像素。但是模型只为三角形的三个顶点指定了u和v坐标,而我们需要为每个像素指定这些坐标。.
By now you can probably see where this is going. Yes, it’s our good friend linear interpolation. We can use attribute mapping to interpolate the values of and across the face of the triangle, giving us at each pixel. From this we can compute , fetch the texel, apply shading, and paint the pixel with the resulting color. You can see the result of doing this in Figure 14-3.
现在你可能已经明白了这是怎么回事。是的,这就是我们的好朋友线性内插。我们可以使用属性映射来插值整个三角形表面的u和v的值,给我们每个像素的(u,v)。由此我们可以计算出(tx,ty),获取texel,应用阴影,并在像素上涂上所得到的颜色。你可以在图14-3中看到这样做的结果。
The results are a little underwhelming. The exterior shape of the crates looks fine, but if you pay close attention to the diagonal planks, you’ll notice they look deformed, as if bent in weird ways. What went wrong?
结果有点不尽人意。箱子的外部形状看起来很好,但如果你仔细注意对角线的木板,你会发现它们看起来是变形的,好像是以奇怪的方式弯曲的。是什么出了问题?
As in Chapter 12 (Hidden Surface Removal), we made an implicit assumption that turns out not to be true: namely, that and vary linearly across the screen. This is clearly not the case. Consider the wall of a very long corridor painted with alternating vertical black and white stripes. As the wall recedes into the distance, the vertical stripes should look thinner and thinner. If we make the coordinate vary linearly with , we get incorrect results, as illustrated in Figure 14-4.
如同第12章(隐藏表面去除),我们做了一个隐含的假设,结果发现并不真实:即u和v在整个屏幕上是线性变化的。情况显然不是这样的。考虑一个很长的走廊的墙壁,上面涂有交替的垂直黑白条纹。当墙壁退到远处时,垂直条纹应该看起来越来越细。如果我们让u坐标随x′线性变化,我们会得到不正确的结果,如图14-4所示。
The situation is very similar to the one we encountered in Chapter 12 (Hidden Surface Removal), and the solution is also very similar: although and aren’t linear in screen coordinates, and are. (The proof is very similar to the proof: consider that varies linearly in 3D space, and substitute and with their screen-space expressions.) Since we already have interpolated values of at each pixel, it’s enough to interpolate and and get and back:
这种情况与我们在第12章(隐藏表面去除)中遇到的情况非常相似,解决方案也非常相似:尽管u和v在屏幕坐标中不是线性的,但u z和v z是线性的。(证明方法与1 z的证明非常相似:考虑u在三维空间中线性变化,用屏幕空间的表达式代替x和y)。因为我们已经有了每个像素的1 z的内插值,所以只需要对u z和v z进行内插,就可以得到u和v。
This produces the result we expect, as you can see in Figure 14-5.
这产生了我们期望的结果,如图14-5所示。
Figure 14-6 shows the two results side by side, to make it easier to appreciate the difference.
图14-6显示了两个并排的结果,以使人们更容易理解其中的差别。
Source code and live demo >>
源代码和现场演示 >>
These examples look nice because the size of the texture and the size of the triangles we’re applying it to, measured in pixels, is roughly similar. But what happens if the triangle is several times bigger or smaller than the texture? We’ll explore those situations next.
这些例子看起来不错,因为纹理的大小和我们要应用的三角形的大小,以像素为单位,大致相似。但如果三角形比纹理大几倍或小几倍会怎样呢?我们接下来会探讨这些情况。
Bilinear Filtering 双线性滤波
Suppose we place the camera very close to one of the cubes. We’ll see something like Figure 14-7.
假设我们把相机放在非常靠近其中一个立方体的地方。我们会看到类似图14-7的东西。
The image looks very blocky. Why does this happen? The triangle on the screen has more pixels than the texture has texels, so each texel is mapped to many consecutive pixels.
图像看起来非常块。为什么会出现这种情况?屏幕上的三角形的像素比纹理的质点要多,所以每个质点被映射到许多连续的像素上。
We are interpolating texture coordinates and , which are real values between and . Later, given the texture dimensions and , we map the and coordinates to and texel coordinates by multiplying them by and respectively. But because a texture is an array of pixels with integer indices, we round and down to the nearest integer. For this reason, this basic technique is called nearest neighbor filtering.
我们正在对纹理坐标u和v进行插值,它们是0.0和1.0之间的实值。之后,给定纹理尺寸w和h,我们将u和v坐标分别乘以w和h,映射为tx和ty的纹理坐标。但由于纹理是一个具有整数索引的像素数组,我们要把tx和ty四舍五入到最近的整数。由于这个原因,这种基本技术被称为近邻过滤。
Even if varies smoothly across the face of the triangle, the resulting texel coordinates “jump” from one whole pixel to the next, causing the blocky appearance we can see in Figure 14-7.
即使(u,v)在整个三角形的表面平滑变化,所产生的texel坐标也会从一个完整的像素点 "跳 "到下一个像素点,导致我们在图14-7中看到的块状外观。
We can do better. Instead of rounding and down, we can interpret a fractional texel coordinate as describing a position between four integer texel coordinates (obtained by the combinations of rounding and up and down). We can take the four colors of the surrounding integer texels, and compute a linearly interpolated color for the fractional texel. This will produce a noticeably smoother result (Figure 14-8).
我们可以做得更好。我们可以把一个小数文元坐标(tx,ty)解释为描述四个整数文元坐标(通过对tx和ty进行四舍五入的组合得到)之间的位置,而不是将tx和ty向下舍入。我们可以从周围的整数texel中提取四种颜色,并为小数texel计算一个线性内插的颜色。这将产生一个明显更平滑的结果(图14-8)。
Let’s call the four surrounding pixels , , , and (for top-left, top-right, bottom-left, and bottom-right, respectively). Let’s take the fractional parts of and and call them and . Figure 14-9 shows , the exact position described by , surrounded by the texels at integer coordinates, and its distance to them.
我们把周围的四个像素称为TL、TR、BL和BR(分别代表左上、右上、左下和右下)。让我们把tx和ty的小数部分称为fx和fy。图14-9显示了C,由(tx,ty)描述的确切位置,被整数坐标的文本所包围,以及它与它们的距离。
First, we linearly interpolate the color at , which is between and :
首先,我们对CT处的颜色进行线性插值,它位于TL和TR之间。
Note that the weight for is , not . This is because as becomes closer to 1.0, we want to become closer to . Indeed, if , then , and if , then .
请注意,TR的权重是fx,而不是(1-fx)。这是因为当fx变得更接近1.0时,我们希望CT变得更接近TR。事实上,如果fx=0.0,那么CT=TL,如果fx=1.0,那么CT=TR。
We can compute , between and , in a similar way:
我们可以用类似的方法来计算TL和TR之间的CB。
Finally, we compute , linearly interpolating between and :
最后,我们计算C,在CT和CB之间进行线性内插。
In pseudocode, we can write a function to get the interpolated color corresponding to a fractional texel:
在伪代码中,我们可以写一个函数来获得对应于小数点的内插颜色。
GetTexel(texture, tx, ty) {
fx = frac(tx)
fy = frac(ty)
tx = floor(tx)
ty = floor(ty)
TL = texture[tx][ty]
TR = texture[tx+1][ty]
BL = texture[tx][ty+1]
BR = texture[tx+1][ty+1]
CT = fx * TR + (1 - fx) * TL
CB = fx * BR + (1 - fx) * BL
return fy * CB + (1 - fy) * CT
}
This function uses floor(), which rounds a number down to the nearest integer, and frac(), which returns the fractional part of a number, and can be defined as x - floor(x).
这个函数使用 floor() ,它将一个数字向下舍入到最接近的整数,以及 frac() ,它返回一个数字的小数部分,可以定义为 x - floor(x) 。
This technique is called bilinear filtering (because we’re doing linear interpolation twice, once in each dimension).
这种技术被称为双线性滤波(因为我们要做两次线性插值,在每个维度上一次)。
Mipmapping Mipmapping
Let’s consider the opposite situation, rendering an object from far away. In this case, the texture has many more texels than the triangle has pixels. It might be less evident why this is a problem, so we’ll use a carefully chosen situation to illustrate.
让我们考虑相反的情况,从远处渲染一个物体。在这种情况下,纹理的质点比三角形的像素多得多。这可能不太明显,为什么这是一个问题,所以我们将用一个精心选择的情况来说明。
Consider a square texture in which half the pixels are black and half the pixels are white, laid out in a checkerboard pattern (Figure 14-10).
考虑一个正方形的纹理,其中一半的像素是黑色的,一半的像素是白色的,呈棋盘式排列(图14-10)。
Suppose we map this texture onto a square in the viewport such that when it’s drawn on the canvas, the width of the square in pixels is exactly half the width of the texture in texels. This means that only one-quarter of the texels will actually be used.
假设我们把这个纹理映射到视口中的一个正方形上,这样当它被画在画布上时,这个正方形的宽度(以像素计)正好是纹理宽度(以特克斯计)的一半。这意味着只有四分之一的特克斯会被实际使用。
We’d intuitively expect the square to look gray. However, given the way we’re doing texture mapping, we might be unlucky and get all the white pixels, or all the black pixels. It’s true that we might be lucky and get a 50/50 combination of black and white pixels, but the 50-percent gray we expect is not guaranteed. Take a look at Figure 14-11, which shows the unlucky case.
从直觉上讲,我们希望这个广场看起来是灰色的。然而,考虑到我们做纹理映射的方式,我们可能不走运,得到所有的白色像素,或者所有的黑色像素。的确,我们可能会很幸运,得到50/50的黑白像素组合,但我们所期望的50%的灰色并不保证。看一下图14-11,它显示了不幸运的情况。
How to fix this? Each pixel of the square represents, in some sense, a texel area of the texture, so we could compute the average color of that area and use that color for the pixel. Averaging black and white pixels would give us the gray we are looking for.
如何解决这个问题?正方形的每个像素在某种意义上代表了纹理的一个2×2 texel区域,所以我们可以计算出该区域的平均颜色,并将该颜色用于像素。对黑白像素进行平均,就可以得到我们想要的灰色。
However, this can get very computationally expensive very fast. Suppose the square is even farther away, so that it’s one-tenth of the texture width. This means every pixel in the square represents a texel area of the texture. We’d have to compute the average of 100 texels for every pixel we want to render!
然而,这可能会很快变得非常昂贵的计算。假设这个广场离得更远,那么它就是纹理宽度的十分之一。这意味着广场上的每个像素都代表了纹理的10×10 texel区域。我们必须为我们想要渲染的每一个像素计算100个特克斯的平均值!这就是为什么我们的计算成本很高。
Fortunately, this is one of those situations where we can replace a lot of computation with a bit of extra memory. Let’s go back to the initial situation, where the square was half the width of the texture. Instead of computing the average of the four texels we want to render for every pixel again and again, we could precompute a texture of half the original size, where every texel in the half-size texture is the average of the corresponding four texels in the original texture. Later, when the time comes to render a pixel, we can just look up the texel in this smaller texture, or even apply bilinear filtering as described in the previous section.
幸运的是,这是一种情况,我们可以用一点额外的内存来代替大量的计算。让我们回到最初的情况,广场是纹理宽度的一半。与其反复计算我们要为每个像素渲染的四个文本的平均值,我们可以预先计算一个原始尺寸的一半的纹理,在这个半尺寸的纹理中,每个文本都是原始纹理中相应的四个文本的平均值。后来,当需要渲染一个像素的时候,我们可以直接在这个较小的纹理中查找文本,甚至可以像上一节中描述的那样应用双线性过滤。
This way, we get the better rendering quality of averaging four pixels, but at the computational cost of a single texture lookup. This does require a bit of preprocessing time (when loading a texture, for example) and a bit more memory (to store the full-size and half-size textures), but in general it’s a worthwhile trade-off.
这样一来,我们就可以得到平均四个像素的更好的渲染质量,但只需要一次纹理查询的计算成本。这确实需要一点预处理时间(例如在加载纹理时)和一点内存(存储全尺寸和半尺寸纹理),但总的来说,这是一个值得权衡的选择。
What about the size scenario we discussed above? We can take this technique further and also precompute one-quarter-, one-eighth-, and one-sixteenth-size versions of the original texture (down to a texture if we wanted to). Then, when rendering a triangle, we’d use the texture whose scale best matches its size and get all the benefits of averaging hundreds, if not thousands, of pixels at no extra runtime cost.
那我们上面讨论的10×大小的情况呢?我们可以进一步利用这一技术,预先计算出原始纹理的四分之一、八分之一和十六分之一的版本(如果我们愿意,还可以计算出1×1的纹理)。然后,在渲染一个三角形时,我们会使用与它的尺寸最匹配的纹理,并在没有额外的运行时间成本的情况下获得平均数百甚至数千像素的所有好处。
This powerful technique is called mipmapping. The name is derived from the Latin expression multum in parvo, which means “much in little.”
这种强大的技术被称为mipmapping。这个名字来自于拉丁语中的multum in parvo,意思是 "小中见大"。
Computing all these smaller-scale textures does come at a memory cost, but it’s surprisingly smaller than you might think.
计算所有这些较小规模的纹理确实需要付出内存成本,但令人惊讶的是它比你想象的要小。
Say the original area of the texture, in texels, is , and its width is . The width of the half-width texture is , but it requires only texels; the quarter-width texture requires texels; and so on. Figure 14-12 shows the original texture and the first three reduced versions.
半宽纹理的宽度是w 2,但它只需要A 4 texels;四分之一宽的纹理需要A 16 texels;以此类推。图14-12显示了原始纹理和前三个缩小的版本。
We can express the sum of the texture sizes as an infinite series:
我们可以把纹理大小的总和表示为一个无限的序列。
This series converges to 4/3, or , meaning that all the smaller textures down to texel only take one-third more space than the original texture.
这个系列收敛于A⋅4/3,即A⋅1.3333,意味着所有小到1×1 texel的纹理只比原始纹理多占用三分之一的空间。
Trilinear Filtering 三线制滤波
Let’s take this one step further. Imagine an object far away from the camera. We render it using the mipmap level most appropriate for its size.
让我们再往前走一步。想象一下,一个物体离摄像机很远。我们使用最适合其尺寸的mipmap级别来渲染它。
Now imagine the camera moves toward the object. At some point, the choice of the most appropriate mipmap level will change from one frame to the next, and this will cause a subtle but noticeable difference.
现在想象一下摄像机向物体移动。在某些时候,最合适的mipmap级别的选择将从一帧改变到下一帧,这将导致一个微妙但明显的差异。
When choosing a mipmap level, we choose the one that most closely matches the relative size of the texture and the square. For example, for the square that was 10 times smaller than the texture, we might choose the mipmap level that is 8 times smaller than the original texture, and apply bilinear filtering on it. However, we could also consider the two mipmap levels that most closely match the relative size (in this case, the ones 8 and 16 times smaller) and linearly interpolate between them, depending on the “distance” between the mipmap size ratio and the actual size ratio.
在选择mipmap级别时,我们选择与纹理和广场的相对大小最接近的级别。例如,对于比纹理小10倍的广场,我们可能会选择比原始纹理小8倍的mipmap级别,并对其应用双线性滤波。然而,我们也可以考虑与相对尺寸最接近的两个mipmap级别(在这种情况下,小8倍和16倍的),并在它们之间进行线性插值,这取决于mipmap尺寸比和实际尺寸比之间的 "距离"。
Because the colors that come from each mipmap level are bilinearly interpolated and we apply another linear interpolation on top, this technique is called trilinear filtering.
因为来自每个mipmap层的颜色都是双线性插值,我们在上面应用另一个线性插值,这种技术被称为三线性过滤。
Summary
In this chapter, we have given our rasterizer a massive jump in quality. Before this chapter, each triangle could have a single color; now we can draw arbitrarily complex images on them.
在这一章中,我们给我们的光栅器带来了质量上的巨大飞跃。在本章之前,每个三角形只能有一种颜色;现在我们可以在上面绘制任意复杂的图像。
我们还讨论了如何确保贴图的三角形看起来不错,无论三角形和贴图的相对大小如何。我们介绍了双线性滤波、mipmapping和三线性滤波,作为解决低质量纹理最常见原因的方法。