Crafting iconic automatic scrolling for Trello

Hey, we're still running a little bit early.

Did anyone have any questions before I get into it about, React Beautiful D& D or Pragmatic Drag and Drop?

If anyone's played with those things, happy to answer any questions.

Yes.

In React Beautiful D& D or Pragmatic Drag and Drop?

Yeah, so just for, I'm just like killing time.

Because we're a little bit early.

But, yeah, so for React Beautiful D& D it has baked in keyboard controls.

Where you effectively press like space bar to lift and then you move, use the arrow keys to move things around, space bar to drop.

What we found though is that was okay, but pretty sub optimal, because screen readers… It was basically trying to get a screen reader to do drag and drop operations.

And so with Pragmatic Drag and Drop, the newer one that we've put out, the intention wasn't to try and get screen readers to perform drag and drop operations.

But rather to get them to perform the same outcomes, you know in a way that's delightful for them.

Yeah, so that's the different approach and we have lots of different controls for how to do it.

But the big idea is not how do you move a card to two positions down using arrows?

It's how do they reprioritize the card using, maybe a menu or a form or something like that.

And that seems to scale really well, and also plays really well with assistive technologies.

Using keyboard controls, if you close your eyes, if you think about drag and drop, and you think about, what would an arrow key actually do if you can't see the screen?

You have no concept of where things even are.

So yeah, moving away from directional controls for the new one.

Cool.

All right, we'll get into it.

Wait, any more questions?

Yes.

No, that's a browser limitation, so yeah, I'm just, I'm still, yeah, you, can a little bit sometimes in some browsers, but according to the spec you can't.

Yeah, pragmatic drag and drop is based on the browser's built in drag and drop capabilities, and unfortunately the spec doesn't allow that.

They like, lock the cursor down, which is unfortunate, because the grab hand looks really good when dragging.

But yeah, if you go on Trello today, they don't actually have the grabbing hand, right?

You'll see they don't, because we don't support it.

I'm sorry, browsers don't support it.

Alright, cool.

Oh yeah, alright.

We'll just get started.

I feel bad though, starting.

Should I start?

Are you guys okay to start?

Yeah.

Obviously you're okay with it.

Would you, would anyone feel bad if I started again if there was like, a few people came in?

Would that be okay?

And if no one else really comes in, we'll just keep going.

Does that sound good to people?

Alright.

More to go then.

Okay, Trello.

Yeah, Trello makes it, really easy and intuitive to drag things around in a column, whether it's cards, you can even drag columns around as well inside a board.

And a big part of what facilitates that intuitive experience is automatic scrolling.

And that's the idea that when you're dragging something close to the edge of a scrollable element we're going to automatically scroll that element on behalf of you.

And this lets you just move things around really quickly and powerfully, right?

And when it all comes together, this is what it looks like in Trello now.

It just feels fantastic, right?

You can move things around.

You can get a lot done really, quick.

And today I'm going to talk about, all the little decisions.

Micro decisions that have gone into making this feel so good.

So firstly for a bit of context, Trello uses an open source library called Pragmatic Drag and Drop, that we released about six months ago.

And Pragmatic Drag and Drop is a wrapper around the browser's built in drag and drop capabilities.

It just makes those safe, easy, and fast to use well.

Cool.

But let's talk, think about like where we're starting.

So this is no automatic scrolling at all.

This is just, if you're using the browser's built in drag and drop capabilities.

What's, that experience like?

When you're dragging a card, in this case, towards the bottom of an element, you'll see that I'm like fiddling around at the bottom there, and it starts to scroll a little bit.

Chrome does have some built in automatic scrolling.

You have to be within a couple of pixels of the edge of the container.

It's very hard to trigger.

You have to hold it over the top of it as well.

Yeah, so not, great.

Unfortunately, there's no way to opt out of this behavior.

So the solution I'm going to talk about, we've worked, within that constraint.

Now, this is, yeah, so there's the little hitboxes there.

They're absolutely tiny.

Firefox also has fairly small hit boxes in there a little bit bigger than Chrome, but not a lot.

You don't get a lot of control on the speed as well.

And Safari, doesn't have, anything at all.

Now, automatic scrolling is not defined in the drag and drop specification.

So Firefox and Chrome have just gone ahead and like just implemented their own thing.

Unfortunately, I don't think it's like fantastic.

But yeah, so that's what kind of what we're working with.

So yeah, let's build up, like, how do we get from what we just saw to something like this?

First step is we need to think about when do we want to start scrolling?

So how far, away from an edge, let's say, this top edge.

How far away before we start scrolling?

And if we make this area too big, it actually gets really hard to work with lists because you can't really move things around the list anymore.

You're just like scrolling it the whole time.

It's pretty unmanageable to use.

Also, if the hitbox is too small, we're back in the same problem we had with, Chrome and Firefox, where those areas are just like absolutely tiny, and you can't really do what you want to do.

Cool.

A formula that we landed on that feels pretty good is either 25 percent of the size of the element, on that axis, or 180 pixels.

Kind of whichever is the smaller of those two values.

Now, this is just totally, values that we, made up.

They seem to feel quite good at scale, though.

So that works really well for, smaller elements.

So over here, we've got a, here we go.

Here we've got, a small element.

You'll see that these hit boxes for the scrolling have shrunk down.

So we have this area in the middle, where automatic scrolling won't happen, and then on the edges, it will.

And then on a large element here, you can see that on this bottom edge, it would have been 215 pixels, but we've clamped it down to 180.

Now, why we do that is because, let's say you've got a really big element.

If you use 25 percent the whole time, sometimes you can just be scrolling something that you didn't anticipate that you would be scrolling.

So yeah, so that's why we clamp it down to 100, 180 pixels.

Cool.

So this is it for a small, element kind of feels pretty good, right?

Using that 20, 25 percent value.

And then for a larger element here, we've clamped it being clamped to 180 pixels.

So you still get hopefully quite a large amount of space in the middle where you can just move things around without scrolling, but still get some generous space for scrolling as well.

So a good question then is how fast should we scroll this thing?

I think the simplest approach, that we could do is just to have one constant max speed in that area.

So we can take a look at what that looks like.

The trouble is if we choose a value that's really slow, it's really clear what's happening here.

You can see where the item is going to go, but it's quite pretty painful if you want to do big movements.

And so if we ramp the speed up, then you have the problem of, it just like shoots away from you when you enter these areas.

So it's really hard to actually, just do anything constructive because you've just blown away your position.

So you're stuck between these two places, like either slower, which is clearer, or faster, which is more powerful, but then you lose all that clarity.

An approach we could then look at is, what if we accelerated this speed, so we'd start with a slower speed at the top, and then as you move down the element, you would get an increasingly, higher speed.

And this actually feels really good, right?

So as you move down the, list, that speed goes higher and higher.

So you start at the top, you have this kind of like gradual build up, you can do more fine controlled movements, and then as you go down, you can just blow past things really quick.

The problem with this approach though, is we're going down, and then all of a sudden it stops.

You're automatically scrolling, great, and then it's finished.

And that's because we're in this sort of acceleration zone, and then when we get beyond the element, we're not over that element anymore, and the browser's hitboxes is all based on elements, and so we're outside of the hitbox of that thing, and we just kill the scroll.

Not ideal.

This current approach is encouraging people who want to go fast, they go down faster and faster then all of a sudden it just stops.

Which is not great.

Yeah, an approach that we went for, which kind of helps with this is to have two bands in this little zone.

So one we're accelerating, we saw that before slower to faster, and then we're having a generous place at the bottom where the speed is like that constant max speed.

And the intention of that is that we're not incentivizing people to continue to pull down anymore because you're not getting any faster, right?

So it's less Incentive to end up down here in that danger zone.

So this is what that looks like in practice So you can see I'm down there in that acceleration kind of window.

I can tweak that Speed by moving it up and down.

And then in the bottom area, they're just moving at that max speed.

We went for 50 percent for both of those.

We can tweak what those sizes are, but 50 percent for both of those seem to feel quite nice at scale for all the different configurations we have out there.

Okay, cool.

So then you might be thinking, okay, instead of having that massive speed drop off, why not, why don't we like have a band of deceleration at the bottom?

So we're going.

Slower, faster, max speed, and then we go, we slow down again and gracefully exit.

Unfortunately, we can't do this because of that, I mentioned that automatic scrolling that the browsers have, which is pretty cooked.

But you can't disable it down the bottom.

So what this ended up being, we should go fast, fast, sorry, faster, max speed, slowing down.

Then you get a little dip of speed right at the end because of the, browser's one, and you can't opt out of.

So yeah, unfortunately couldn't do that.

So this is what is there today.

This is, seems to be the nice, like compromise, working within the constraints of the web platform.

Another thought we might then want to think is, okay, what should that acceleration look like when I'm in that area, right?

That acceleration area.

And my initial thought was, why don't we make this like exponential, right?

So start slow.

And as you get towards the end, it ramps up the speed.

And this is what that looks like.

And it is hard to tell on the slides.

But what we found is that it doesn't actually feel that good using a exponential kind of growth like that.

And the reason is, I think, because we have each pixel that you're moving in that band has a disproportionate amount of impact on the total speed.

On this part here, you're moving, those first pixels and it's not really changing the speed much at all.

And then I've just told not to use the spotlight.

So I will try not to use that.

Although I may still use it because I really need it.

And then on the end, we've got a really big impact in speed.

So those last few pixels, it's like the speed's like changing heaps.

So you don't really have a consistency in how you, your, what your cursor's doing.

So what I actually ended up feeling really good was to make it just linear.

So thinking of that acceleration band as like a range input, ideally in a range input, every pixel has the same impact as the pixel before it.

And so this is really nice.

So you get really fine grained control of what that acceleration should be, because every pixel has the same impact on the net speed.

So this is what that looks like.

So now every, as you move up and down, you are having a predictable impact on the speed.

Pretty sweet.

Problem.

We have a problem with what we have so far, and that is speed spikes.

So let's say we have a column here, and we've got a card near the bottom, and we want to drag this card around.

Now right now that card is in our max scroll speed area.

So if we lift it, what's going to happen is it's just going to shoot at that max speed, right?

Which is pretty disjointing.

You're just trying to drag something around and all of a sudden you've lost context.

You don't even know what's happening.

And the page is shot away from you.

This problem, occurs in a few other places as well.

So this is, we're entering a column from the side.

We get that spike as well.

We've entered into that max speed kind of area, and this is the same thing, but just, entering from one list and into another, we get that we're getting these big spikes in speed, which is quite disorientating.

Hard to know kinda what's happening.

So the solution to this problem is time.

And what we do is, when you're over an element, we start a timer.

And that, as that time, that timer will depress the amount of speed that's available in that frame for a scroll.

So as that timer runs down, it will allow the speed to increase.

I'll show you what that looks like.

So here is a list and I'm going to lift this card, 10 at the bottom here.

And I'm using color to hopefully demonstrate what's going to happen.

I've also slowed things down a bit so that hopefully it's, hopefully you can see what's happening.

But as I'm lifting, you, will hopefully see that the Excel, the card, the acceleration slowly increases over time.

So this, we avoid that problem of the list just like shooting away from you.

It allows you to grab that card and move it up if you want and you're not going to lose all that position.

This is lifting something in the middle of a list.

So we start that timer like as soon as we're over it.

So what, it's actually quite quick in production.

So when you lift it in the middle of the column and you move down, you're not even going to feel this time dampening.

Like it's very subtle, it just fades away quite quickly.

But the idea is that even though, even if you're lifting in the middle, you're not going to feel inhibited by it.

This is it, but entering into the side of a list.

So yeah, you can see the timer starts again when we enter into that list.

And we never get any massive speed spikes, right?

We're like.

We're never allowing the scroll to just absolutely shoot away.

And this is what it looks like, moving between two lists.

So we never, hopefully it's always really clear what's happening to the user.

Cool.

So next thing I'm going to talk about is overflow scrolling.

And we've talked about this, problem of we have this really nice scrollable list, but we fall off the bottom of it, right?

And we just lose all that speed.

And we have the same problem here, and this is a board.

I've just, so that purple border is a scrollable, element.

I'm trying to scroll it here, like to the left and the right.

But as soon as I go off the side of the element, I wasn't able to scroll it anymore.

So we've created an optional add on for the Autoscroller package that we have, which allows for overflow scrolling.

So this is, you can now see that for this list, I'm able to continue to scroll that list even after I've left it, which, honestly feels awesome.

It just feels like a superpower when you're doing this, like in Trello, or in other products that are using it.

Yeah, that you can continue to scroll that list after you've left it.

It just feels so good.

And the time dampening also applies in that overflow area.

So we start the timer the, the timer for the time that I'm pinning is shared between those two zones, which is really, cool.

So you're not going to get any speed spikes, even when you're going in that kind of overflow area.

Yeah, and this is what it looks like.

Yeah, it's so good.

So good.

I love this.

Just go play with it in Trello.

It feels really good.

Yes, this is it for the board as well.

So I've got the light green area is the, there's the acceleration darker green is max speed.

And that red is the overflow.

So now we can pull on both sides of the board and it feels like you can just pull the board around.

Feels so good.

Okay, now this is actually awesome.

Okay, on the right hand edge here, you might notice, I'm going to use the pointer, I'm sorry, I'm going to use the pointer.

Alright, up here, at the top and the bottom.

Oh, you guys can't see it, I can't see it.

Oh, I'm so sorry.

I've wasted your lives.

Yeah, I'm, I apologize profusely.

There you go.

So up here, and down here, what we've actually done is you have the ability to specify for an edge, so in this case the right edge, I actually want to be able to scroll that edge above and below that edge.

Which sounds abstract, but I'll show you what it actually looks like.

So it means now, you can scroll that right edge when you're above it.

Which feels absolutely awesome, right?

So you can be over here and you're like pulling it to the right.

Just because you're close to that edge.

And what I'm showing off here is that it has the same acceleration on that axis as those bands, so when I'm in the light green band, I'll go back and show you that again, when you're in the light green band, it's going to have the same acceleration on that whole kind of cross axis for the right band.

So it all just feels very like fluid and smooth as you're moving around.

And when you put it all together, damn, this is so cool.

Like it actually feels really good.

I don't know if you've used Trello lately, but you should.

And it's just, it feels really great.

Yeah.

So you can just pull this thing around.

It absolutely, it's, it, you can get so much.

You can.

Move things around really well.

Okay.

Now there is a problem with overflow scrolling and people may have spotted it already.

And I'm going to show it to you here.

I'm slowing everything down.

It's going to be a bit hard.

I think the light makes it a bit tough.

Boom.

You might see it.

I'm going to, you'll see it there.

There's that gap.

That's going up when I'm in that overflow area.

So the trouble is because we're no longer over that…sorry, let me go back a step.

What we're doing is when you drag over a card we're inserting a shadow to show where it's going to drop now when we're no longer over the list We're technically not dragging over any of those cards anymore.

Like we're outside the list.

And so those cards like card 14, 15, 16, 17 they're not saying hey I'm being dragged over anymore because they're not like you're outside the list.

And so even though you're down here like you think you might be dropping at the end of the list, but actually You'll be dropping up here.

Now we maintain that spot by using a thing called stickiness in pragmatic drag and drop, which I won't go into, but yeah, that is cool.

But yeah, so that is a problem, right?

And that's why we've made the overflow scroll optional because it has a drawback, right?

Like it's not going to be perfect.

We've accepted that for Trello.

It actually feels really great.

And you can solve this problem really quickly by just dragging your card up slightly back into the list and then you're back in the right spot.

We could compensate for that technically, but it would be a lot of code.

You start to work against the benefits we were unlocking by using the web platforms drag and drop in the first place.

So yeah, we've just made it overflow scrolling like optional.

It's a way that… So I mean we've talked a lot about scrolling But how do we even know what element we should scroll in a given frame?

What do we how do we do that?

What are we gonna do?

So generally what happens is the overflow scroller will on every frame it looks up where the users pointer is based on the events we're getting grabs the element under that and then like searches from there to try and figure out what to scroll and it does that on every single frame.

That's really nice because it means that no matter what you're over no matter what happens what changes it's just gonna figure It out.

And a really good first question is how do we even know that an element is scrollable in the first place?

So the first approach you could do is you could start where the user's current, pointer is, and you look, you just keep looking up the elements.

And you basically have to call window dot getComputedStyles, which is not cheap, on an element, and you pull out the overflow, overflowY, or overflowX value, and check if any of them are set to scroll or auto, and then that thing is potentially scrollable.

But, obviously not ideal, very expensive, you have to, on every single frame, you are calling you know, a lot of this quite expensive function and the deeper you go into DOM trees, obviously the more expensive it's going to get.

So not ideal.

What we've done for this auto scroller is instead to use registrations, which is really nice.

The whole kind of like library uses this.

So you explicitly register 'Hey, I want this scrollable element to be a, we want to have auto scroll here'.

And that's makes things really nice because then, when you're starting here, you don't need to search up through the whole tree to search up to the first… to the next registration and we add a data attribute to the elements that have auto scroll enabled on it.

And so just look, jump straight up to that and then it can just jump straight to the next one.

And if there's no more, it's done.

So this is really a nice way of doing things.

Using registrations also has the benefit of, they can conditionally enable or disable, scrolling, while it's happening.

When it's all automatic trying to figure out what's happening in the browser.

It becomes very difficult to do opt out behavior as well.

Cool.

Yeah.

When we, when, so we've got this, we want to do some automatic scrolling, we're doing it on every frame.

What elements should we scroll?

So generally the algorithm is quite similar to how browsers work by default.

So if you have two scrollable elements, we're going to scroll the inner one first.

Until it's exercised finished all its scrolling and then we'll go up to the parent So here I've got two nested things inside here.

It's, you can, there are the two hitboxes and yeah, so this is a little thing I made it's not super accurate.

But yeah, that's the idea So you're scrolling in one first than the other one.

So how the algorithm works, which we've made is that on a given frame only one element can be scrolled.

You can only scroll something vertically and something horizontally.

That could be the same element, it could be different elements.

But you can't scroll, two things vertically.

So what does that mean?

It means that in scenarios where, A and B, if A can scroll horizontally, we can, keep searching up for something that might want to search.

Sorry, to scroll horizontally.

So yeah, you get something like that.

Where you can like scroll on both axes.

And what that looks like in Trello, I've made these things visible, is so you can scroll the board at the same time as you can scroll the list.

Because the lists are the inner one, they're being scrolled vertically, and the outer one being scrolled horizontally.

Yeah, and this is it without the hitboxes visible.

Yeah, things are a little bit different for overflow scrolling.

So that's when you're in this kind of red areas down here.

So the algorithm is a little bit different because we can't rely on the DOM order anymore.

These are just like artificial boxes that we've created.

So it is less performant, right?

Because on every frame we actually have to go, okay, everything that's registered overflow scrolling, we have to get, calculate all these hit boxes and then figure out if you're over any of them.

So yeah, overflow scrolling is awesome, but it's just like use sparingly in this.

I think in Trello we only have.

Basically columns and the board have it, it's fine, but yeah, so we have to, because there's no like hierarchy in these things, we just have to get all the boxes and see which ones, you're over.

For, this is what that looks like, I think, here we go, yeah, so I am in the overflow area for both of them, and yeah, so I'm scrolling the, columns horizontally and the board vertically in that overflow scroll area.

Yeah, so the lightning round of other things that I don't have time to go into, but are also super cool.

So we actually account for different, frame rates.

So this is a, 60 FPS running versus 128 frames per second.

So we adjust the total speed that you're scrolling, so on faster devices, it's not going to just like absolutely rock it away.

So we didn't pick a, like a frame per second value.

It's actually a, sorry, a pixels per frame.

It's actually pixels per second that we're targeting.

We break that down into how many milliseconds have passed since the last frame, and so we can adjust the speed so that we get a consistent speed across those two things.

This was super fun to build.

I built a little FPS monitor in the browser.

And for slower devices, so this one's 11 frames per second.

But 20, 30, 40.

These are super reasonable things to find in the wild.

Especially if your browser's under load.

You've got something happening on your machine.

What we do for those ones is we actually, we don't use that variable frame rate, we actually cap the amount of change that can happen to 16 pixels for a frame.

And so that means the speed itself is slower, but hopefully it's still clear what's happening.

If we went for a, the other approach we saw, we would actually get massive jumps, in, in scroll, which wouldn't be great.

So all this is to say that it feels good on lower frame rate devices as well.

We have, this is a board that's being scrolled horizontally at a, our standard speed.

We actually give you the option to specify one of two presets.

We couldn't pick for certain things.

It does feel like for bigger elements on certain conditions, having a faster speed does make sense.

So this is the same thing, but with a bit faster.

So this is what actually we have in Trello today.

So you can specify that top speed a bit as well.

And this is those two side by side.

So you can see.

I think the top one's slower, it's faster.

Yeah, this is just a little thing, the automatic scroller, technically you don't even need it until the drag even starts at all if you don't want, so you can just lazy load the whole thing.

This, that was on 4G, it happened pretty quick, it's absolutely tiny.

But yeah, the idea is that, if you want to like absolutely squeeze every dropout of performance, you can just load, load in the autoscroller later.

That's fine.

You don't need it straight away.

This is it on my like standard connection.

It's yeah, it's so fast.

It's like you don't even see the, oh no, that's the 4G.

There you go.

Sorry, the other one was my other one.

This is 4G.

Got the slides mixed up.

That's okay.

Yeah, so yeah.

Load super quick if you want to load it later, you know as well.

Yeah, so Pragmatic drag and drop, yeah, open source library that, you're welcome to use in your own stuff as well.

It just makes the web platform's drag and drop easy to use and safe and fast, like it's all broken up into little pieces, so you just use the pieces you want and nothing else.

Our autoscroller is an optional package for that, and overflow scrolling is an optional piece of that as well.

But the nice part about that model is if you don't want any autoscrolling, you don't have to pay any code for it, don't worry about it.

And if what I said doesn't make sense for you or your experience, like you don't have to even use our autoscroller, you could make your own, that's totally fine.

Or if browsers get their act together, we can just not use this one and use theirs like yeah, so that's really a really nice model to do things Yeah, pragmatic drag and drops actually, just so you know, like it's not, it is powering Trello today, which is great.

But if you pretty much drag and drop anything in Jira and Confluence as well, like they're using it as well.

And all the stuff that I talked about in terms of that autoscroller, they're using that as well.

So we designed it to be like, hopefully flexible enough to handle all these cases really well.

We've come a super long way and I was like, yesterday afternoon, I was like really hyped listening to this song.

So I put together like just a little video of, it all coming together.

Yeah, it's awesome.

But, yeah, like playing with it, it's really fun, actually.

It feels, in the end, it comes together really well.

You can find me now, as of yesterday, I think, on Blue Sky.

Which, yeah, so I'm new.

Say hello.

And, yeah, what else was I going to say?

Oh, yeah, I am hoping to post, so I had, I made a little, standalone Trello for this talk.

It's not quite ready to share with people because it's got heaps of, as you can imagine, like comments in there and debugging stuff for all those visuals that I hope to take out and yeah I'll put it out there for people who want to see how it all kind of works.

Thanks very much.