It can outpaint

#10
by Nothingnamed - opened

Seems you've accidentally made it capable of outpainting. tested with turbo.

Click to view images

1 (40)

ComfyUI_00001_

1 (26)

ComfyUI_00010_

image

Oh, nice discovery!
Does it work reliably?
Thanks for letting me know :)

It's semi reliable more or less, But sometimes like when i tried it on the scream painting it refused to extend it and just stretch the image, I assume cause it might know it too well. Or sometimes with a square portrait trying to make torse/lower body it might just end up effectively trimming the sides, that might also have been a well know thing i think i tried that with the girl with the pearl earing.
It also warps things here and there and sometimes it want to change things more. Seems it also does it better if you only extend in one dimension at a time.

Now krea2 does have some ability to outpaint without it with some usage of the encodeqwen node, i assume the krea2 edit encode does basically the same as that one, combined with some latent masking and differential diffusion.

also posted the workflow to civitai https://civitai.com/models/2772215/krea-2-outpaint

So at the very least you got good springboard to jump from to make it into a proper lora for such things.

Always nice to use things for more than the intended purpose by playing around. great lora.

deleted
edited Jul 12

It's semi reliable more or less, But sometimes like when i tried it on the scream painting it refused to extend it and just stretch the image, I assume cause it might know it too well. Or sometimes with a square portrait trying to make torse/lower body it might just end up effectively trimming the sides, that might also have been a well know thing i think i tried that with the girl with the pearl earing.

Hang on... why not just ask the instruction model to zoom out the image?
Why outpaint to begin with?
I didn't develop this, but I think the very prompt itself should be able to handle it.

Hang on... why not just ask the instruction model to zoom out the image?
Why outpaint to begin with?
I didn't develop this, but I think the very prompt itself should be able to handle it.

To be honest i didn't even think about since i was coming from trying to do it before i knew of the lora.

But testing it, while it does work it causes more distortions and changes in the image doing it like that with different aspect ratios.
And this workflow also seem to work without prompts. but such instructions can certainly help it along as well.

It's semi reliable more or less, But sometimes like when i tried it on the scream painting it refused to extend it and just stretch the image, I assume cause it might know it too well. Or sometimes with a square portrait trying to make torse/lower body it might just end up effectively trimming the sides, that might also have been a well know thing i think i tried that with the girl with the pearl earing.

Hang on... why not just ask the instruction model to zoom out the image?
Why outpaint to begin with?
I didn't develop this, but I think the very prompt itself should be able to handle it.

There is a technical difference about zooming out and out-painting the image. zooming out shall move the pixels of the original subject into the center by recreating the subject,which may cause fidelity loss and change the pixel.While on the other hand out-painting in practice shall pin the original pixels in place without modification in ideal conditions and generate the missing parts around. That is why they are different.

@conradlocke Thanks for the lora and workflow. it is not a replacement of a real editor model however it is capable of doing some stuff. I like how you managed to come up with a solution. Looking for the next version. Can the next version have more resolution like in V1 for the grounding pixels? Also a variety of aspect ratios please in the dataset as in landscape and portrait to prevent the double generation and squeezing.

It's semi reliable more or less, But sometimes like when i tried it on the scream painting it refused to extend it and just stretch the image, I assume cause it might know it too well. Or sometimes with a square portrait trying to make torse/lower body it might just end up effectively trimming the sides, that might also have been a well know thing i think i tried that with the girl with the pearl earing.
It also warps things here and there and sometimes it want to change things more. Seems it also does it better if you only extend in one dimension at a time.

Now krea2 does have some ability to outpaint without it with some usage of the encodeqwen node, i assume the krea2 edit encode does basically the same as that one, combined with some latent masking and differential diffusion.

also posted the workflow to civitai https://civitai.com/models/2772215/krea-2-outpaint

So at the very least you got good springboard to jump from to make it into a proper lora for such things.

Always nice to use things for more than the intended purpose by playing around. great lora.

i wonder if you utilized the conditioning rebalance node if it would allow for outpainting the scream image? it pushes NSFW im sure it would bypass that in the filter

Seems it can but you need the prompting. Doesn't work promptless.
Even without the latent reference it wants to recreate the scream so it know it really well.

Click to view images

krea2_0210

It really wants to duplicate.
krea2_0220
But with describing what's where in the new area really helps.
krea2_0221
same as above but without the reference latent demonstrating just how well it know it from the conditioning. which of course have less seams since it is just using the condition. As opposed to unknown images where significant details would change, specially on people.
krea2_0223
You don't really need my nonsense
krea2_0249

In many cases if you try to force it not to change it will create seams or oddly connect so it's not ideal for perfect outpainitng, but if you allow it to do it's thing it's smooth. You don't really need much more than the default workflow for the lora in most cases. It's quite a good model for many uses.

The main important thing to make the lora work better at different aspect ratios and sizes even when using two reference seems to be scaling and padding the reference latents to match the size of the output. It can also help at times to pad in the image conditioning.

Sign up or log in to comment