Coordinate space for Line endpoints in "Flat 2D" geometry mode?

Geordie Hall

4 years ago
updated 4 years ago
Answered

Hi there,

I've been trying to use the Line component with a "screen-space camera" UI canvas (with canvas scaler), and have noticed some skewing of line end caps with the "Billboard" line type.

Switching to the "Flat 2D" geometry type resolves the end cap skewing, but then the endpoints are in a different place.

I tried converting the previous world-space endpoint coordinates to the canvas' local space (eg. canvas.transform.InverseTransformPoint(StartPoint)) and to screenspace coordinates (eg. RectTransformUtility.WorldToScreenPoint(canvas.worldCamera, StartPoint) ), and turning off the canvas scaler, but couldn't seem to get anything to line up properly.

I couldn't see anything in the docs about the coordinate spaces needed so thought I'd just check if there's something I'm missing?

Cheers!

Reporting a bug? please specify Unity version:

2019.4

Reporting a bug? please specify Shapes version:

2.0.0

Reporting a bug? please specify Render Pipeline:

Pinned replies

Freya Holmér creator

4 years ago
Answer
Answered

thanks for the report! just found the issue with the skewing, and it has now been fixed, so it's coming in the next update!

as for getting correct coordinates, you likely want to use parent.InverseTransformPoint( worldPos ) to get the local coords you should use for the component, where parent is the parent transform of the shape

if you don't want to wait for the next release, you can make these changes in Line 2D Core.cginc

half3 localZ = normalize( half3( UNITY_MATRIX_M[0].z, UNITY_MATRIX_M[1].z, UNITY_MATRIX_M[2].z ) );
normal = cross( tangent, localZ );

Reply Link

Replies 5

Newest first
Newest first Oldest first

4 years ago

ah, also I just realized I got it wrong! it shouldn't be parent.inverse, it should be the shape.transform.inversetransformpoint

and yeah, the way rect transforms work is a bit of a weird case, not entirely sure how it plays with scale and how that affects transformation operations

Thanks for the fix, have applied it now (although I think what I was seeing might have actually just been perspective-related skewing from using the Billboard mode - the "Flat 2D" mode is definitely what I wanted).

I tried using local coords with parent.InverseTransformPoint() as suggested but things still didn't line up properly, so I made a new scene and just set everything up in the hierarchy and it worked fine, so started looking for differences (I'm spawning these components at runtime for the real game).

First thing I noticed was that the game object I was spawning didn't have a rect transform even though it was under a canvas (I'm coming to Unity from UE4, so not used to all of its weird edges). But even spawning it properly so it had a rect transform didn't fix, but I noticed it had a massive z value. So then I tried zeroing the local position after setting the parent.. and that actually seemed to work!

Not sure if the z component gets ignored in certain cases when using rect transforms, but maybe just a newbie mistake on my part. Glad you at least found a bug while investigating :)

Thanks!

The CanvasScaler seems to alter the scale of the canvas GameObject, but everything under it between the canvas and the line has scale of 1

4 years ago
Under Review

all coordinates should be in local space - do you have any transform scale applied to any of the transforms in its chain of parents?