We have,

$$ \begin{aligned} I(x)= & \int \frac{x^2\left(x \sec ^2 x+\tan x\right)}{(x \tan x+1)^2} d x \\\\ = & x^2 \int \frac{x \sec ^2 x+\tan x}{(x \tan x+1)^2} d x \\\\ & \quad-\int\left\{\frac{d}{d x}\left(x^2\right) \int \frac{x \sec ^2 x+\tan x}{(x \tan x+1)^2} d x\right\} d x \text { (integration by parts) } \end{aligned} $$

$$ =x^2\left(\frac{-1}{x \tan x+1}\right)+\int \frac{2 x}{x \tan x+1} d x $$

Now, let

$$ \begin{aligned} I_1 & =2 \int \frac{x}{x \tan x+1} d x \\\\ & =2 \int \frac{x \cos x}{x \sin x+\cos x} d x \end{aligned} $$

$$ \begin{aligned} & \text { On putting } x \sin x+\cos x=t \\\\ & \Rightarrow (x \cos x+\sin x-\sin x) d x=d t \\\\ & \Rightarrow x \cos x d x=d t \end{aligned} $$

$$ \begin{aligned} &\therefore I_1 =2 \int \frac{d t}{t}=2 \log t+c \\\\ & =2 \log (x \sin x+\cos x)+c \end{aligned} $$

$$ \begin{gathered} \therefore I(x)=\frac{-x^2}{x \tan x+1}+2 \log(x \sin x+\cos x)+c \end{gathered} $$

When, $x=0$, then

$$ \begin{array}{ll} & I(0)=0+2 \log (1)+c=0 \\\\ &\Rightarrow c=0 \\\\ &\therefore I(x)=\frac{-x^2}{x \tan x+1}+2 \log (x \sin x+\cos x) \end{array} $$

$$ \begin{aligned} &\therefore I(x) =\frac{-x^2}{x \tan x+1}+2 \log (x \sin x+\cos x) \\\\ &\Rightarrow I\left(\frac{\pi}{4}\right) =\frac{-\frac{\pi^2}{16}}{\frac{\pi}{4}+1}+2 \log \left(\frac{\pi}{4} \times \frac{1}{\sqrt{2}}+\frac{1}{\sqrt{2}}\right) \\\\ & =\log \left(\frac{(\pi+4)^2}{32}\right)-\frac{\pi^2}{4(\pi+4)} \end{aligned} $$