...

If F1 and F2 are 2 forces, then maximum resultant force (Fmax) = F1+F2

 

Hence, Fmax = 10 = F1+F2

Also, Fmin = 6 = F1-F2

Adding the 2 equations, 

16 = 2F1

F1 = 8N

F2 = 2 N 

 

Now, each force is increased by 3N

Hence, F1' = 8+3 = 11N

F2' = 2+3= 5N 

 

Now, they are acting at 90 degree to one another, hence theta = 90

 

Fres = sqrt(F1'^2 + F2'^2 + 2F1' * F2' cos(theta))

Fres = sqrt(11^2 + 5^2 + 2*11* 5 cos(90))

Fres = sqrt(121+25)

Fres = sqrt(146)

Fres = 12.08 N