This code imports necessary modules for extracting content from a website, interacting with a web browser, and summarizing the extracted content. A function testExtractor is then defined, which uses Selenium to extract and summarize content from a webpage and returns the summary.

Run example

npm run import -- "test article summarizer"

test article summarizer

const extractArticle = importer.import("extract llm article")
const getClient = importer.import("selenium client")
const summerizeArticle = importer.import("summarize llm article")

async function testExtractor(startPage) {
  if (!startPage) {
    startPage = 'https://tsakraklides.com/2025/02/05/in-the-age-of-infinite-consumer-choice-the-only-choice-is-collapse/'
  }

  driver = await getClient()

  try {

    let result = await extractArticle(driver, startPage)

    driver.quit()

    let summary = await summerizeArticle(result)

    return summary
  } catch (e) {
    driver.quit()

    throw e
  }
}


module.exports = testExtractor

'use strict';

const { extractLlmArticle } = require('./extractLlmArticle');
const { seleniumClient } = require('./seleniumClient');
const { summarizeLlmArticle } = require('./summarizeLlmArticle');

/**
 * Extracts a summary from a given article using a Selenium client and returns it.
 * 
 * @param {string} startPage The starting page URL to extract from.
 * @param {Object} options Optional configuration for the client.
 * @returns {Promise} The extracted summary.
 */
async function testExtractor(startPage, options = {}) {
  // Set default start page if not provided
  if (!startPage) {
    startPage = 'https://tsakraklides.com/2025/02/05/in-the-age-of-infinite-consumer-choice-the-only-choice-is-collapse/';
  }

  // Initialize Selenium client
  const driver = await seleniumClient(options);

  try {
    // Extract article content
    const result = await extractLlmArticle(driver, startPage);

    // Clean up client
    await driver.quit();

    // Summarize extracted article
    const summary = await summarizeLlmArticle(result);

    return summary;
  } catch (e) {
    // Clean up client on error
    await driver.quit();

    // Re-throw error
    throw e;
  }
}

module.exports = testExtractor;

Code Breakdown

Importing Modules

const extractArticle = importer.import('extract llm article')
const getClient = importer.import('selenium client')
const summarizeArticle = importer.import('summarize llm article')

• Three modules are imported from an importer module:
  • extractArticle: responsible for extracting content from a website using an LLM article extraction function.
  • getClient: returns a Selenium client, which is used to interact with a web browser.
  • summarizeArticle: summarizes the content extracted by the extractArticle function.

Test Extraction Function

async function testExtractor(startPage) {
  //...
}

• The testExtractor function is an asynchronous function that takes an optional startPage parameter.
• If startPage is not provided, it defaults to a specific URL.

Setting up Selenium Client

driver = await getClient()

• A Selenium client is obtained using the getClient function and stored in the driver variable.

Extracting and Summarizing Article

try {
  let result = await extractArticle(driver, startPage)
  driver.quit()
  let summary = await summarizeArticle(result)
  return summary
} catch (e) {
  driver.quit()
  throw e
}

• The extractArticle function is called with the driver and startPage to extract content from the website.
• The extracted content is stored in the result variable.
• The Selenium client is quit.
• The extracted content is summarized using the summarizeArticle function and stored in the summary variable.
• If an error occurs during this process, the error is caught and the Selenium client is quit before re-throwing the error.

Exporting the Function

module.exports = testExtractor

• The testExtractor function is exported as a module, making it available for use in other JavaScript files.