(sorry my bad english)

What is solution for this problem?
I could not solve this problem up to the end

My not finished solution:

P(&x, &y)
{
s = x + y;
for (int i = 0; i < s; ++ i)
{
x0 = x;
y0 = y;
x = y0 - 2*x0*x0;
y = (int)(sqrt (2) * x0) // floor
}
}

but I don't know what to do further.

Tell me please.