Help with getting the full screenshot, aiquery, and/or scrape of entire page.

[scaruslooner]

I am having trouble getting the full screenshot/info of the entire page (check screenshots). I also include my code which i tried to go to the last page but it doesn't work either. How do i get aiquery to get the full answer, scrape or take full screenshot of the entire site?

import { test } from './fixture';
import { expect } from '@playwright/test';
import * as fs from 'fs';
import * as path from 'path';

test('interact with Microsoft Copilot', async ({ page, ai, aiQuery }) => {
  // Set a reasonable viewport size
  await page.setViewportSize({ width: 1280, height: 800 });

  // Navigate to Microsoft Copilot
  await page.goto('https://copilot.microsoft.com/chats');
  
  // Wait for the page to be fully loaded
  await page.waitForLoadState('networkidle');

  // Prepare the prompt message
 // ... existing code ...
 const prompt = `give me 4 recipes`;
 // ... existing code ...
  try {
    // Find and click the message input area, then enter the prompt
    //await ai('Click the message input area');
    await ai(`Enter entire prompt which is "${prompt}" at once in the input area then hit enter, then wait 15 seconds`);
    
    await page.keyboard.press('End');
    // Additional wait to ensure content settles
    await page.waitForTimeout(2000);

    // Take screenshots of both the visible viewport and full page
    const screenshotDir = path.join(process.cwd(), 'screenshots');
    if (!fs.existsSync(screenshotDir)) {
      fs.mkdirSync(screenshotDir);
    }

    const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
    const pathway = path.join(screenshotDir, `full-page-${timestamp}.png`);

    // Take a full page screenshot
    await page.screenshot({path: pathway, fullPage: true});


    // Get the complete response after scrolling
    const response = await aiQuery(`string, get the entire response text from Copilot's last message, making sure to include all content even if it requires page down. Look for text within elements with class 'text-message-content' or similar message containers.`);
    
     // Create output directory if it doesn't exist
     const outputDir = path.join(process.cwd(), 'output');
     if (!fs.existsSync(outputDir)) {
       fs.mkdirSync(outputDir);
     }
 
     // Save response to file with timestamp
     const timestamp2 = new Date().toISOString().replace(/[:.]/g, '-');
     const outputPath = path.join(outputDir, `copilot-response-${timestamp2}.txt`);
     fs.writeFileSync(outputPath, response);
     
     // Log status
     console.log('Copilot Response:', response);
     console.log('Response saved to:', outputPath);
     
     // Verify we got a substantial response
     expect(response).toBeTruthy();
     expect(response.length).toBeGreaterThan(100); // Ensure we got more than just a greeting
     
   } catch (error) {
     console.error('Error during Copilot interaction:', error);
     throw error;
   }
});

[yuyutaotao]

Hi @scaruslooner
⁠You may change the viewport size to include more content. But usually there is a size limitation for the image input to LLM. So currently it may be a little bit hard to analyze the screenshot of the entire site.

[yuyutaotao]

We will find a way to pass the entire html to LLM, so that it can "see" more data outside the screen.