Connecting to an external vespa instance in docker compose

Support
1

You have advised that one should not use the integrated vespa instance in production. I have therefore set up a docker-compose file according to the configuration advice here: https://docs.marqo.ai/2.11/Guides/Advanced-Usage/configuration/#configure-backend-communication

  marqo:
    container_name: 'marqo-vespa'
    image: 'marqoai/marqo:2.11'
    restart: always
    depends_on:
      node0:
        condition: service_healthy
    environment:
      VESPA_CONFIG_URL: "http://node0.vespanet:19071"
      VESPA_DOCUMENT_URL: "http://node2.vespanet:8080"
      VESPA_QUERY_URL: "http://node3.vespanet:8081"
    ports:
      - '8882:8882'
    networks:
      - vespanet

The vespa nodes share the network vespanet and the vespa configserver is up and healthy.
The result however is:

$ docker logs marqo-vespa
External vector store configured. Using external vector store
Starting Marqo throttling
Called Marqo throttling start command
Marqo throttling is now running
ERROR:marqo.tensor_search.on_start_script:Failed to bootstrap vector store. If you are using an external vector store, ensure that Marqo is configured properly for this. See https://docs.marqo.ai/2.11/Guides/Advanced-Usage/configuration/ for more details. Error: Application 'default.default' has no active session.

I presume I am missing some additional configuration parameters. Can you point out my error?

(And apologies for double-posting, I seem to have created this post as a message first, now as a topic)

2

Hello @rcking. Thanks for experimenting the docker compose solution to set up Marqo in production. Could you also share the config of Vespa in your docker-compose file? Is the Vespa running and have you deployed an application package to it?

Alternatively, please feel free to check GitHub - marqo-ai/marqo-on-kubernetes for ways to deploy Marqo with an external Vespa using k8s. Cheers.

3

My apologies, I should have posted the full docker-compose file from the start. I has been adapted from this vespa example.

services:

  # configserver
  node0:
    container_name: node0
    hostname: node0.vespanet
    image: vespaengine/vespa
    networks:
      - vespanet
    command: configserver,services
    healthcheck:
      test: curl http://localhost:19071/state/v1/health
      timeout: 10s
      retries: 3
      start_period: 40s
    ports:
      - 19071:19071
      - 19100:19100
      - 19050:19050
      - 20092:19092
    environment:
      VESPA_CONFIGSERVERS: node0.vespanet
      VESPA_CONFIGSERVER_JVMARGS: "-Xms32M -Xmx128M"
      VESPA_CONFIGPROXY_JVMARGS: "-Xms32M -Xmx32M"
  
  # admin server
  node1:
    container_name: node1
    hostname: node1.vespanet
    image: vespaengine/vespa
    networks:
      - vespanet
    command: services
    depends_on:
      node0:
        condition: service_healthy
    ports:
      - 19098:19098
      - 20095:19092
    environment:
      VESPA_CONFIGSERVERS: node0.vespanet

  # feed server
  node2:
    container_name: node2
    hostname: node2.vespanet
    image: vespaengine/vespa
    networks:
      - vespanet
    command: services
    depends_on:
      node0:
        condition: service_healthy
    ports:
      - 8080:8080
      - 20096:19092
    environment:
      VESPA_CONFIGSERVERS: node0.vespanet

  # query server
  node3:
    container_name: node3
    hostname: node3.vespanet
    image: vespaengine/vespa
    networks:
      - vespanet
    command: services
    depends_on:
      node0:
        condition: service_healthy
    ports:
      - 8081:8080
      - 20097:19092
    environment:
      VESPA_CONFIGSERVERS: node0.vespanet

  # content server
  node4:
    container_name: node4
    hostname: node4.vespanet
    image: vespaengine/vespa
    networks:
      - vespanet
    command: services
    depends_on:
      node0:
        condition: service_healthy
    ports:
      - 19107:19107
      - 20100:19092
    environment:
      VESPA_CONFIGSERVERS: node0.vespanet

  marqo:
    container_name: 'marqo-vespa'
    image: 'marqoai/marqo:2.11'
    restart: always
    depends_on:
      node0:
        condition: service_healthy
    environment:
      VESPA_CONFIG_URL: "http://node0.vespanet:19071"
      VESPA_DOCUMENT_URL: "http://node2.vespanet:8080"
      VESPA_QUERY_URL: "http://node3.vespanet:8081"
    ports:
      - '8882:8882'
    networks:
      - vespanet

networks:
  vespanet:
    driver: bridge
    name: vespanet

The configserver is up and healthy:

$ curl http://localhost:19071/state/v1/health
{
  "time" : 1724223560382,
  "status" : {
    "code" : "up"
  },
  "metrics" : {
    "snapshot" : {
      "from" : 1.724223500093E9,
      "to" : 1.724223560089E9
    },
    "values" : [ {
      "name" : "requestsPerSecond",
      "values" : {
        "count" : 2,
        "rate" : 0.03333555570371358
      }
    }, {
      "name" : "latencySeconds",
      "values" : {
        "average" : 0.002,
        "sum" : 0.004,
        "count" : 2,
        "last" : 0.002,
        "max" : 0.011,
        "min" : 0.002,
        "rate" : 0.03333555570371358
      }
    } ]
  }
}

But - I have deployed no application package. I assumed that the marqo does this automatically at start-up, I guess I am mistaken about that?

5

OK, my key understanding is that the “basic” vespa application configuration must be established before starting marqo.

The marqo class index_management.py then downloads and modifies this configuration, addiing all of the necessary marqo-specific schema.

I solved the issue (I think! At least I was able to start marqo and create and populate a new index) by creating an emphermal container that starts after the vespa configserver and loads the application configuration, whereby marqo only starts after this container has successfully closed.

6

Hello @rcking . Sorry for the late reply. Your understanding is correct. We only auto deploy an application package if you use the embedded Vespa in Marqo. For external Vespa, you will need to deploy an application package first. The reason is that Marqo does not have enough information to config Vespa nodes in hosts.xml and services.xml file. I’ll update the documents to clarify this. Thanks for exploring and sharing your findings.