POB

Public Opportunities Bot

×Table of Contents
POB
Open Source Project

POB

POB (Public Opportunities Bot) is an open source framework that automates the capture, parsing, indexing and publishing of public sector contract opportunities.

The source code for this open project may be found on GitHub.

Introduction

What is POB

POB (Public Opportunities Bot) is an open source framework (written in Java 11) that automates the capture, parsing, indexing and publishing of public sector contract opportunities. At the time of writing, most public sector procurement frameworks in the UK lack public APIs or the ability to subscribe to notifications when new opportunities are published that can then be integrated into existing sales systems. POB automatically captures and parses new commercial contract opportunities from relevant public sector procurement frameworks and can be configured to then index and publish these opportunities to any target software service.

POB Slack Plublisher

Extending POB

POB can be easily extended to parse any procurement framework and publish to any target software service. Please see Develop your own POB Parser and Develop your own POB Publisher respectively.

Deploying POB

POB can either be run locally or deployed to the cloud, such as to the Microsoft Azure or AWS cloud computing platforms. Please see Run POB Locally and Deploy POB to the Cloud respectively.

1. Download POB

1.1. Requirements

Please ensure that the following prerequisite software services are installed in your local development environment.

1.2. Cloning

Please clone the Git repository for this project into your local development environment by executing the following commands via the Git command line (or equivalent). The directory into which you clone this Git repository will hereafter be referred to as POB_BASE.


git clone https://github.com/hyperlearningai/pob
cd pob
1.3. Maven Modules

POB employs a Maven parent-child module hierarchy that can be imported into your chosen IDE (for example Eclipse or VSCode) by simply importing POB_BASE as a Maven project. The following Maven modules make up the POB project:

  • pob-app - Spring Boot application to run POB locally or deployed as a (cloud-based) webapp with an in-built configurable CRON scheduler
  • pob-common - Common configuration used across the project including pob.yaml
  • pob-function-app - Spring Cloud Function for running POB as a (cloud-based) serverless function app using an in-built configurable CRON scheduler
  • pob-jpa - Java Persistence API repositories and configuration
  • pob-model - Framework and opportunity models
  • pob-parsers - Abstract framework parser class and native framework parser implementations
  • pob-publishers - Abstract publisher class and native publisher implementations
  • pob-service - Service to run all parsers and publishers
  • pob-utils - Common utility methods such as date formatting
1.4. Configuration

All configuration for POB can be found in the pob-common Maven module, and specifically in src/main/resources/conf/pob.yaml. The following sections describe the various configuration namespaces, properties and example values.

It is recommended that passwords, secrets and other sensitive information such as JDBC and Slack Webhook URLs be set as environment variables in your relevant local development and/or cloud-based deployment environments in order to avoid having this information in pob.yaml (or similar property files) which may accidentally be made visible to other users via version control repositories or other means. By default, pob.yaml is configured to read sensitive property values from environment variables.

Namespace: application

PropertyDescriptionExample Values
server.portThe HTTP port number on which to serve the POB Spring Boot application.8080
scheduler.cronCRON expression controlling the POB scheduler when run as a Spring Boot application. By default, POB will parse the relevant frameworks for new opportunities every 20th minute.0 0/20 * * * *


Namespace: application.frameworks

As of version 1.0.0, POB provides the following native framework parsers:

If you extend POB by creating your own custom POB parser, as detailed in Develop your own POB Parser, then a new framework configuration object must be added to the array of procurement frameworks in the application.frameworks namespace.

PropertyDescriptionExample Values
idUnique string identifier for the frameworkdos
nameHuman-readable name for the framework'Digital Outcomes and Specialists Framework'
enabledWhether the framework should be processed by POBtrue
baseUrlThe URL to the framework itselfhttps://www.digitalmarketplace
.service.gov.uk
opportunitiesUrlThe URL that should be used by the parser to parse opportunitieshttps://www.digitalmarketplace
.service.gov.uk/digital-outcomes-and-specialists/opportunities
parserFully
QualifiedClassName
The fully-qualified class name of the custom parserai.hyperlearning.pob.parsers.dos
.DosLatestOpportunitiesParser
filterWhether to apply a keyword filter when parsing opportunities from this frameworktrue
keywordsSpace-delimited list of keywords to use if filtering is enableddata software


Namespace: application.publishers

As of version 1.0.0, POB provides the following native publishers:

  • Slack - Communication and collaboration platform

If you extend POB by creating your own custom POB publisher, as detailed in Develop your own POB Publisher, then a new publisher configuration object must be added to the array of publishers in the application.publishers namespace.

PropertyDescriptionExample Values
idUnique string identifier for the publisherslack
enabledWhether the publisher should be processed by POBtrue
publisherFully
QualifiedClassName
The fully-qualified class name of the custom publisherai.hyperlearning.pob.publishers
.slack.SlackPublisher
propertiesCustom key-value objects that will be injected into the custom publisher classchannel: pob
webhook: https://...


Namespace: application.storage.sql

POB uses a SQL database to persist parsed framework opportunities.

For local development and testing purposes, HSQLDB provides a lightweight in-memory relational database management system that can be configured to persist to the local filesystem if required. The drivers for HSQLDB are already included in the POB application.

PropertyDescriptionExample Values
driverClassNameJDBC Driver Class Nameorg.hsqldb.jdbc.JDBCDriver
jdbcUrlJDBC Connection Stringjdbc:hsqldb:file:/tmp/pob/pob-db
usernameDatabase User'sa'
passwordDatabase User Password''


2. Run POB

2.1. Setting up Slack

If you have enabled the native POB Slack publisher (enabled by default), then you will need to create a Slack App for your Slack Workspace and configure an incoming webhook in order to enable parsed opportunities to be posted to your chosen Slack channel. To do this, and assuming that you have a Slack Workspace where you are provisioned rights to create and install apps, navigate to https://api.slack.com/apps and follow the process as described below:

  1. Select 'Create New App'.
  2. Select 'From scratch' in the resultant popup dialog.
  3. Give your app a name (e.g. pob) and pick the relevant workspace.
  4. Select your app from the list at https://api.slack.com/apps
  5. In the 'Basic Information' settings and under 'Display Information', provide a short description and icon for your new app.
  6. In the 'Incoming Webhooks' settings, turn on 'Activate Incoming Webhooks'.
  7. Then under 'Webhook URLs for your Workspace', select 'Add New Webhook to Workspace'
  8. Select the Slack Channel that you wish POB to post parsed opportunities to, and select 'Allow'
  9. Going back to 'Webhook URLs for your Workspace', copy the newly created Webhook URL into pob.yaml under slack.properties.webhook. Also set the name of the Slack channel under slack.properties.channel (or set the relevant environmental variables should you wish to avoid putting secrets into property files).

POB is now configured to automatically publish parsed opportunities to the chosen Slack channel using the incoming webhook.

2.2. Run POB Locally

To run POB locally, a Spring Boot application (that will run all enabled parsers and publishers, currently synchronously, managed by a configurable CRON schedule), is provided in the pob-app Maven module. Run ai.hyperlearning.pob.apps.PobApp found in this Maven module to run POB, served on a HTTP port number and managed by a CRON schedule both of which can be configured in pob.yaml - see Configuration above. By default, the POB Spring Boot application will run on port 8080 and parse relevant procurement frameworks for new opportunities every 20th minute.

The POB Spring Boot application can be deployed to the cloud as a webapp. However this is not recommended - a more efficient and significantly less expensive way to run POB in the cloud is to run it as a serverless Function app managed and triggered by a CRON schedule. This is described in further detail below.

2.3. Deploy POB to the Cloud

To deploy and run POB using a cloud computing platform such as Microsoft Azure or AWS, a Spring Cloud Function is provided in the pob-function-app Maven module. As the name suggests, the Spring Cloud Function project enables you to develop serverless functions in Java and focus on their business logic agnostic of the target runtime environment (i.e. whether Azure Functions or AWS Lambda for example). In this instance, a POB Spring Cloud function is available that will run POB as a timer-based serverless function in the cloud, making it much less expensive to run POB compared to deploying POB as an always-on webapp.

To run POB via a Microsoft Azure Function app, an Azure Handler of the Spring Cloud function is natively provided. To deploy POB to an Azure Function, create an empty Azure Function app via the Azure portal and then set the relevant Azure subscription and resource properties required by pom-function-app/pom.xml, detailed below, as environment variables in your development environment or CI/CD pipeline configuration:


<!-- Module Properties -->
<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <deployment.azure.function.subscriptionId>${env.POB_AZURE_FUNCTION_SUBSCRIPTION_ID}</deployment.azure.function.subscriptionId>
    <deployment.azure.function.resourceGroup>${env.POB_AZURE_FUNCTION_RESOURCE_GROUP_NAME}</deployment.azure.function.resourceGroup>
    <deployment.azure.function.appName>${env.POB_AZURE_FUNCTION_NAME}</deployment.azure.function.appName>
    <deployment.azure.function.region>${env.POB_AZURE_FUNCTION_REGION}</deployment.azure.function.region>
    <start-class>ai.hyperlearning.pob.apps.functions.PobFunctionApp</start-class>
</properties>

Note that any environment variable placeholders in pob.yaml will have to be configured as application settings in your Azure Function app. To set these application settings, navigate to your Azure Function app via the Azure Portal and select 'Configuration' then 'Application settings'.

Then to deploy POB to an Azure Function, please install the Azure CLI and Azure Functions Core Tools 3.x, and execute the following command line statements:


# Build POB
cd $POB_BASE
mvn clean
mvn package

# Deploy POB as an Azure Function
cd pob-function-app
az login
mvn azure-functions:deploy


Whether running POB locally or as a cloud-based serverless function, POB will run all enabled parsers (currently synchronously) managed by the configurable CRON schedule configured in pob.yaml and persist new opportunities to the POB SQL database. Once persisted, unpublished opportunities will be published using all enabled publishers. The screenshot below shows a new opportunity available via the DOS framework that has been published to a Slack channel by POB.

POB Slack Plublisher

3. Extend POB

3.1. Develop your own POB Parser

POB can be easily extended to parse any procurement framework. To create your own custom POB parser in Java, just extend the ai.hyperlearning.pob.parsers.OpportunityParser abstract class found in the pob-parsers Maven module and implement the parse() method that must return Set<Opportunity>. Then add a new framework object to the application.frameworks namespace in pob.yaml and POB will automatically run your custom parser as part of its pipeline.


import ai.hyperlearning.pob.parsers.OpportunityParser;

public class CustomOpportunitiesParser extends OpportunityParser {

    public CustomOpportunitiesParser(Framework framework) {
        super(framework);
    }

    @Override
    public Set<Opportunity> parse() {
        ...
    }

}


To view the Opportunity model, please review ai.hyperlearning.pob.model.Opportunity found in the pob-model Maven module. And to see an example POB parser, please review ai.hyperlearning.pob.parsers.dos.DosLatestOpportunitiesParser (DOS framework parser) or ai.hyperlearning.pob.parsers.cf.CfLatestOpportunitiesParser (Contracts Finder framework parser), both of which are provided natively by POB and which may be found in the pob-parsers Maven module.

Native POB DOS framework parser


package ai.hyperlearning.pob.parsers.dos;

import java.time.LocalDate;
import java.util.LinkedHashSet;
import java.util.Set;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import ai.hyperlearning.pob.model.Framework;
import ai.hyperlearning.pob.model.Opportunity;
import ai.hyperlearning.pob.parsers.OpportunityParser;
import ai.hyperlearning.pob.utils.DateFormattingUtils;
import ai.hyperlearning.pob.utils.StringUtils;

/**
 * Custom Opportunity Parser - DOS Framework
 *
 * @author jillurquddus
 * @since 0.0.1
 */

public class DosLatestOpportunitiesParser extends OpportunityParser {

    private static final Logger LOGGER = LoggerFactory.getLogger(
            DosLatestOpportunitiesParser.class);
    private static final String SEARCH_RESULTS_CSS_QUERY = 
            "li.app-search-result";
    private static final String GENERAL_CSS_QUERY = "ul.govuk-list";
    private static final String TITLE_CSS_QUERY = "a.govuk-link";
    private static final String SUMMARY_CSS_QUERY = "p.govuk-body";
    private static final String BUYER_TEXT_PREFIX = "Organisation: ";
    private static final String DATE_PUBLISHED_TEXT_PREFIX = "Published: ";
    private static final String DATE_CLOSING_TEXT_PREFIX = "Closing: ";

    public DosLatestOpportunitiesParser(Framework framework) {
        super(framework);
    }

    @Override
    public Set<Opportunity> parse() {

        Set<Opportunity> opportunities = new LinkedHashSet<Opportunity>();
        String opportunitiesUrl = null;
        try {

            // Construct the URL to GET with any required keyword filtering
            if ( getFramework().isFilter() && !getFramework()
                    .getKeywords().isBlank() ) {
                String keywordsFilterRequestParams = StringUtils
                        .keywordsToGetRequestParams(
                                getFramework().getKeywords());
                opportunitiesUrl = getFramework().getOpportunitiesUrl()
                        .replace("?q=&", 
                                "?q=" + keywordsFilterRequestParams + "&");
            } else
                opportunitiesUrl = getFramework().getOpportunitiesUrl();

            // Get the HTML contents of the DOS Opportunities URL
            LOGGER.debug("Parsing {}", opportunitiesUrl);
            Document document = Jsoup.connect(opportunitiesUrl).get();

            // Parse and iterate opportunity search results
            Elements searchResults = document.select(SEARCH_RESULTS_CSS_QUERY);
            for (Element searchResult : searchResults) {

                // Parse URI (normally the opportunity URL if applicable)
                String uri = getFramework().getBaseUrl() + searchResult
                        .selectFirst(TITLE_CSS_QUERY).attr("href");

                // Parse title
                String title = searchResult.selectFirst(
                        TITLE_CSS_QUERY).text();

                // Parse buyer
                String buyer = searchResult.selectFirst(GENERAL_CSS_QUERY)
                        .selectFirst("li").text()
                        .replace(BUYER_TEXT_PREFIX, "")
                        .strip();

                // Parse summary
                String summary = searchResult.selectFirst(SUMMARY_CSS_QUERY)
                        .text();

                // Parse date published
                String datePublishedString = searchResult
                        .select(GENERAL_CSS_QUERY)
                        .get(2).selectFirst("li").text()
                        .replace(DATE_PUBLISHED_TEXT_PREFIX, "")
                        .strip();
                LocalDate datePublished = DateFormattingUtils
                        .EEEEddMMMMyyyyToLocalDate(datePublishedString);

                // Parse date closing
                String dateClosingString = searchResult
                        .select(GENERAL_CSS_QUERY)
                        .get(2).select("li").get(2).text()
                        .replace(DATE_CLOSING_TEXT_PREFIX, "")
                        .strip();
                LocalDate dateClosing = DateFormattingUtils
                        .EEEEddMMMMyyyyToLocalDate(dateClosingString);

                // Create an opportunity object and add it to the set
                Opportunity opportunity = new Opportunity(uri, uri, 
                        getFramework(), title, buyer, summary, 
                        datePublished, dateClosing);
                opportunities.add(opportunity);

            }

        } catch (Exception e) {
            LOGGER.error("Unable to parse the DOS Opportunities URL", e);
        }

        return opportunities;

    }

}


Native POB Contracts Finder framework parser


package ai.hyperlearning.pob.parsers.cf;

import java.time.LocalDate;
import java.util.LinkedHashSet;
import java.util.Map;
import java.util.Set;

import org.jsoup.Connection;
import org.jsoup.Jsoup;
import org.jsoup.Connection.Method;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import ai.hyperlearning.pob.model.Framework;
import ai.hyperlearning.pob.model.Opportunity;
import ai.hyperlearning.pob.parsers.OpportunityParser;
import ai.hyperlearning.pob.utils.DateFormattingUtils;

/**
 * Custom Opportunity Parser - Contracts Finder Service
 *
 * @author jillurquddus
 * @since 0.0.1
 */

public class CfLatestOpportunitiesParser extends OpportunityParser {

    private static final Logger LOGGER = LoggerFactory.getLogger(
            CfLatestOpportunitiesParser.class);
    private static final String FORM_TOKEN_CSS_QUERY = "input[name=form_token]";
    private static final String SEARCH_RESULTS_CSS_QUERY = 
            "div.search-result";
    private static final String GENERAL_CSS_QUERY = 
            "div.search-result-entry";
    private static final String BUYER_CSS_QUERY = 
            "div.search-result-sub-header";
    private static final String SUMMARY_CSS_QUERY = 
            "div.wrap-text";
    private static final String DATE_PUBLISHED_TEXT_PREFIX = 
            "Publication date";
    private static final String DATE_CLOSING_TEXT_PREFIX = 
            "Closing";

    public CfLatestOpportunitiesParser(Framework framework) {
        super(framework);
    }

    @Override
    public Set<Opportunity> parse() {

        Set<Opportunity> opportunities = new LinkedHashSet<Opportunity>();
        try {

            // Get the form token and cookies from the search form page
            Connection.Response formPageResponse = Jsoup.connect(
                    getFramework().getOpportunitiesUrl())
                    .method(Method.GET)
                    .execute();
            Document formPageDocument = formPageResponse.parse();
            Map<String, String> formPageCookies = formPageResponse.cookies();
            String formPageToken = formPageDocument
                    .select(FORM_TOKEN_CSS_QUERY)
                    .first()
                    .attr("value");
            LOGGER.debug("Parsed form token {}", formPageToken);

            // Submit a POST request with the relevant form 
            // data, form token, cookies and keyword filtering
            String keywords = null;
            if (getFramework().isFilter() && 
                    !getFramework().getKeywords().isBlank())
                keywords = getFramework().getKeywords();
            else
                keywords = "";
            Connection.Response formPostResponse = Jsoup.connect(
                    getFramework().getOpportunitiesUrl())
                    .header("Content-Type", 
                            "application/x-www-form-urlencoded;charset=UTF-8")
                    .data("keywords", keywords)
                    .data("location", "all_locations")
                    .data("postcode", "")
                    .data("postcode_distance", "5")
                    .data("open", "1")
                    .data("tender", "1")
                    .data("planning", "1")
                    .data("speculative", "1")
                    .data("public_notice", "1")
                    .data("supplychain_notice", "1")
                    .data("published_from", DateFormattingUtils
                            .getCurrentDateString("dd/MM/yyyy"))
                    .data("sort", "notices.cf_published_date:DESC")
                    .data("form_token", formPageToken)
                    .data("adv_search", "")
                    .cookies(formPageCookies)
                    .method(Method.POST)
                    .execute();

            // Parse the search results
            Document formPostResponseDocument = formPostResponse.parse();
            Elements searchResults = formPostResponseDocument.select(
                    SEARCH_RESULTS_CSS_QUERY);
            for (Element searchResult : searchResults) {

                // Parse URI (normally the opportunity URL if applicable)
                String uri = searchResult.selectFirst("h2")
                        .selectFirst("a").attr("href");

                // Parse title
                String title = searchResult.selectFirst("h2")
                        .selectFirst("a").text();

                // Parse buyer
                String buyer = searchResult.selectFirst(BUYER_CSS_QUERY).text();

                // Parse summary
                String summary = searchResult.select(SUMMARY_CSS_QUERY)
                        .get(1).text();

                // Parse search result entries for additional metadata
                String datePublishedString = null;
                String dateClosingString = null;
                Elements searchResultEntries = searchResult
                        .select(GENERAL_CSS_QUERY);
                for (Element searchResultEntry : searchResultEntries) {
                    String text = searchResultEntry.text();
                    if ( text.contains(DATE_PUBLISHED_TEXT_PREFIX) )
                        datePublishedString = text.replace(
                                DATE_PUBLISHED_TEXT_PREFIX, "").strip();
                    else if ( text.contains(DATE_CLOSING_TEXT_PREFIX) )
                        dateClosingString = text.replace(
                                DATE_CLOSING_TEXT_PREFIX, "").strip();
                }

                // Format date published
                LocalDate datePublished = datePublishedString != null ? 
                        DateFormattingUtils.ddMMMMyyyyToLocalDate(
                                datePublishedString) : null;

                // Format date closing
                LocalDate dateClosing = dateClosingString != null ? 
                        DateFormattingUtils.ddMMMMyyyyToLocalDate(
                                dateClosingString.split(",")[0]) : null;

                // Create an opportunity object and add it to the set
                Opportunity opportunity = new Opportunity(uri, uri, 
                        getFramework(), title, buyer, summary, 
                        datePublished, dateClosing);
                opportunities.add(opportunity);

            }

        } catch (Exception e) {
            LOGGER.error("Unable to parse the Contracts Finder "
                    + "Opportunities URL", e);
        }

        return opportunities;

    }

}
3.2. Develop your own POB Publisher

POB can be easily extended to publish to any target software service. To create your own custom POB publisher in Java, just extend the ai.hyperlearning.pob.publishers.OpportunityPublisher abstract class found in the pob-publishers Maven module and implement the publish(Opportunity opportunity) method that must return a boolean value indicating whether publication of the given opportunity object has been successful or not. Then add a new publisher object to the application.publishers namespace in pob.yaml and POB will automatically run your custom publisher as part of its pipeline. Note that any properties you define for your custom publisher in pob.yaml will be injected into and made available to your custom Java publisher as long as it extends the OpportunityPublisher abstract class.


import ai.hyperlearning.pob.publishers.OpportunityPublisher;

public class CustomPublisher extends OpportunityPublisher {

    public CustomPublisher(Map<String, Object> properties) {
        super(properties);
    }

    @Override
    public boolean publish(Opportunity opportunity) {
        ...
    }

}


To see an example POB publisher, please review ai.hyperlearning.pob.publishers.slack.SlackPublisher (Slack publisher) provided natively by POB and which may be found in the pob-publishers Maven module.

Native POB Slack Publisher


package ai.hyperlearning.pob.publishers.slack;

import java.io.IOException;
import java.util.Map;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpStatus;

import ai.hyperlearning.pob.model.Opportunity;
import ai.hyperlearning.pob.publishers.OpportunityPublisher;
import ai.hyperlearning.pob.utils.StringUtils;
import okhttp3.Call;
import okhttp3.MediaType;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;

/**
 * Custom Opportunity Publisher - Slack Publisher
 *
 * @author jillurquddus
 * @since 0.0.1
 */

public class SlackPublisher extends OpportunityPublisher {

    private static final Logger LOGGER = 
            LoggerFactory.getLogger(SlackPublisher.class);

    private static final String CHANNEL_PROPERTY_KEY = "channel";
    private static final String WEBHOOK_PROPERTY_KEY = "webhook";
    private static final int MESSAGE_DASH_CHARACTER_LENGTH = 100;
    private static final String UNKNOWN_TEXT_VALUE = "Unknown";
    private static final String JSON_PLACEHOLDER_SLACK_CHANNEL = 
            "[SLACK_CHANNEL]";
    private static final String JSON_PLACEHOLDER_OPPORTUNITY_TITLE = 
            "[OPPORTUNITY_TITLE]";
    private static final String JSON_PLACEHOLDER_OPPORTUNITY_BUYER = 
            "[OPPORTUNITY_BUYER]";
    private static final String JSON_PLACEHOLDER_OPPORTUNITY_FRAMEWORK_NAME = 
            "[OPPORTUNITY_FRAMEWORK_NAME]";
    private static final String JSON_PLACEHOLDER_OPPORTUNITY_DATE_PUBLISHED = 
            "[OPPORTUNITY_DATE_PUBLISHED]";
    private static final String JSON_PLACEHOLDER_OPPORTUNITY_DATE_CLOSING = 
            "[OPPORTUNITY_DATE_CLOSING]";
    private static final String JSON_PLACEHOLDER_OPPORTUNITY_SUMMARY = 
            "[OPPORTUNITY_SUMMARY]";
    private static final String JSON_PLACEHOLDER_OPPORTUNITY_URL = 
            "[OPPORTUNITY_URL]";
    private static final String POST_REQUEST_JSON_BODY_TEMPLATE = 
            "{" + 
                    "\"channel\": \"" + JSON_PLACEHOLDER_SLACK_CHANNEL + "\", " +
                    "\"text\": \"" + 
                        "-".repeat(MESSAGE_DASH_CHARACTER_LENGTH) + "\\n" + 
                        "*New Opportunity: " + JSON_PLACEHOLDER_OPPORTUNITY_TITLE + "*\\n" +
                        "-".repeat(MESSAGE_DASH_CHARACTER_LENGTH) + "\\n" + 
                        "*Buyer:* " + JSON_PLACEHOLDER_OPPORTUNITY_BUYER + "\\n" + 
                        "*Framework:* " + JSON_PLACEHOLDER_OPPORTUNITY_FRAMEWORK_NAME + "\\n" + 
                        "*Date Published:* " + JSON_PLACEHOLDER_OPPORTUNITY_DATE_PUBLISHED + "\\n" + 
                        "*Date Closing:* " + JSON_PLACEHOLDER_OPPORTUNITY_DATE_CLOSING + "\\n" + 
                        "*Summary:* " + JSON_PLACEHOLDER_OPPORTUNITY_SUMMARY + "\\n" + 
                        "*Link:* " + JSON_PLACEHOLDER_OPPORTUNITY_URL + "\\n\\n" + 
                        "↓" + 
                    "\", " + 
                    "\"unfurl_links\": true" + 
            "}";

    public SlackPublisher(Map<String, Object> properties) {
        super(properties);
    }

    @SuppressWarnings("deprecation")
    public boolean publish(Opportunity opportunity) {

        // Parse publisher properties
        String channel = (String) getProperties().get(CHANNEL_PROPERTY_KEY);
        String webhook = (String) getProperties().get(WEBHOOK_PROPERTY_KEY);

        // Create the message as a JSON object
        String json = POST_REQUEST_JSON_BODY_TEMPLATE
                .replace(JSON_PLACEHOLDER_SLACK_CHANNEL, channel)
                .replace(JSON_PLACEHOLDER_OPPORTUNITY_TITLE, 
                        StringUtils.cleanJsonValueString(
                                opportunity.getTitle()))
                .replace(JSON_PLACEHOLDER_OPPORTUNITY_BUYER, 
                        StringUtils.cleanJsonValueString(
                                opportunity.getBuyer()))
                .replace(JSON_PLACEHOLDER_OPPORTUNITY_FRAMEWORK_NAME, 
                        StringUtils.cleanJsonValueString(
                                opportunity.getFramework().getName()))
                .replace(JSON_PLACEHOLDER_OPPORTUNITY_DATE_PUBLISHED, 
                        opportunity.getDatePublished() != null ? 
                                opportunity.getDatePublished().toString() : 
                                    UNKNOWN_TEXT_VALUE)
                .replace(JSON_PLACEHOLDER_OPPORTUNITY_DATE_CLOSING, 
                        opportunity.getDateClosing() != null ? 
                                opportunity.getDateClosing().toString() : 
                                    UNKNOWN_TEXT_VALUE)
                .replace(JSON_PLACEHOLDER_OPPORTUNITY_SUMMARY, 
                        StringUtils.cleanJsonValueString(
                                opportunity.getSummary()))
                .replace(JSON_PLACEHOLDER_OPPORTUNITY_URL, 
                        opportunity.getUrl() != null ? 
                                opportunity.getUrl() : UNKNOWN_TEXT_VALUE);

        // Create the HTTP client and POST request object
        OkHttpClient client = new OkHttpClient();
        RequestBody body = RequestBody.create(
                  MediaType.parse("application/json"), json);
        Request request = new Request.Builder()
                  .url(webhook)
                  .post(body)
                  .build();

        // Submit the POST request and get the Slack webhook response
        Response response = null;
        try {

            LOGGER.debug("POSTing opportunity to Slack channel: \n{}", json);
            Call call = client.newCall(request);
            response = call.execute();
            return (response.code() == HttpStatus.OK.value()) ? true : false;

        } catch (IOException e) {

            LOGGER.error("Error encountered when publishing to Slack", e);
            return false;

        } finally {

            // Close the response object to avoid connection leaks
            if (response != null) {
                try {
                    response.close();
                } catch (Exception e) {

                }
            }

        }

    }

}

4. Further Information

4.1. License

POB is distributed under the MIT license.

4.2. Version History
VersionChangelog
1.0.0
  • Added native Digital Outcomes and Specialists (DOS) framework parser
  • Added native Contracts Finder parser
  • Added native Slack publisher


4.3. Roadmap
  • Add a native Elasticsearch publisher
  • Add a native API
  • Add further public sector procurement frameworks
4.4. Credits

The following open source software frameworks and libraries are used by POB.

4.4. Contact

For further information and guidance, please contact: