SaFi Bank Space : Install Meiro Platform (Sandbox using docker-compose)

Created by Fol Justin Lacsina, last modified on Sep 16, 2022

Content Sections

  1. Specs

  2. Meiro Installation
    a. Integration (MI)
    b. Business Explorer (CDP)
    c. Events (ME)
    d.Opensearch
    e. CockroachDB
    f. Terraform

Spec Requirements (Bare minimum)

Whole basic stack consists of 5 apps (7 servers in total + 1 load balancer)

  • Integrations - 4cores, 8G ram, 160G SSD

  • Business Explorer - 4cores, 8G ram, 160G SSD

  • Events - 2cores, 4G ram, 160G SSD

  • OpenSearch - 4cores, 8G ram, 160G SSD

  • CockroachDB(3x) - 4cores, 8G ram, 160G SSD

  • Operating system - Image debian-11-bullseye-v20220822

  • Docker Registry Secrets: VAULT-PATH: secrets/cicd/manual/meiro-docker-registry

Install Meiro Integration (MI)

source code:https://github.com/meiroio/mi_production_environment
VM specs - 4cores, 8G ram, 160G SSD

  1. Setup Firewall access to port 22, 80 and 443

  2. ssh into the VM make use correct Specs and OS

  3. Install prerequisites.

    sudo apt install git python3-pip docker-compose

sudo usermod -aG docker $USER

newgrp docker

  4. Docker Login
    NOTE: have your VM or NAT external IPs whitelisted

    docker login images.meiro.solutions
u: <<fetch in vault>>
p: <<fetch in vault>>

  5. Clone the repository

    git clone git@github.com:meiroio/mi_production_environment.git

cd mi_production_environment/

  6. Run Script (change the value of parameters)
    This will pull container images, check necessary requirements, create yaml template config

    ./update_production_environment.sh --https \
--domain=mi.example.com \                          #DNS should be pointing to external IP for letsecnrypt chanllege
--docker-registry-url='images.meiro.solutions' \.  #docker registry login
--docker-registry-user='myusername' \              #get actual value in vault
--docker-registry-password='mypassword' \          #get actual value in vault
--client-name="SaFiBank-Demo" \                    #branding and metadata
--workers-count=3                                  #worker count

  7. Run Docker Compose
    After Step 5, docker-compose.yml have been created

    docker-compose up

visit https://mi.example.com

  8. Check for Errors
    - If you have SSL errors, make sure --domain is resolvable externally, attached the IP and rerun 5 & 6
    - Cant pull? check 3 is done already, make sure you have your IPs whitelisted to the Meiro Registry
    - Intermittent pull error? Check the network limitation of your Nat gateway or VMS
    https://cloud.google.com/nat/docs/ports-and-addresses#:~:text=Cloud%20NAT%20adjusts%20the%20minimum%20ports%20per%20VM%20instance%20if%20necessary

Install Meiro Business Explorer (CDP)

source code:https://github.com/meiroio/cdp_production_environment
VM specs - 4cores, 8G ram, 160G SSD

  1. Setup Firewall access to port 22, 80 and 443

  2. ssh into the VM make use correct Specs and OS

  3. Install prerequisites.

    sudo apt install git python3-pip docker-compose

sudo usermod -aG docker $USER

newgrp docker

  4. Docker Login
    NOTE: have your VM or NAT external IPs whitelisted

    docker login images.meiro.solutions
u: <<fetch in vault>>
p: <<fetch in vault>>

  5. Clone the repository

    git git@github.com:meiroio/cdp_production_environment.git

cd cdp_production_environment/

  6. Run Script (change the value of parameters)
    This will pull container images, check necessary requirements, create yaml template config

    ./update_production_environment.sh --https \
--domain=cdp.example.com \                   #DNS should be pointing to external IP for letsecnrypt chanllege
--client-name="SaFiBank-Demo"                #branding and metadata

  7. Run Docker Compose
    After Step 5, docker-compose.yml have been created

    docker-compose up

visit https://cdp.example.com

  8. Check for Errors
    - If you have SSL errors, make sure --domain is resolvable externally, attached the IP and rerun 5 & 6
    - Cant pull? check 3 is done already, make sure you have your IPs whitelisted to the Meiro Registry
    - Intermittent pull error? Check the network limitation of your Nat gateway or VMS
    https://cloud.google.com/nat/docs/ports-and-addresses#:~:text=Cloud%20NAT%20adjusts%20the%20minimum%20ports%20per%20VM%20instance%20if%20necessary

Install Meiro Events (ME)

source code: https://github.com/meiroio/me_production_environment

VM spec - 4cores, 8G ram, 160G SSD

  1. Setup Firewall access to port 22, 80 and 443

  2. ssh into the VM make use correct Specs and OS

  3. Install prerequisites.

    sudo apt install git python3-pip docker-compose

sudo usermod -aG docker $USER

newgrp docker

  4. Docker Login
    NOTE: have your VM or NAT external IPs whitelisted

    docker login images.meiro.solutions
u: <<fetch in vault>>
p: <<fetch in vault>>

  5. Clone the repository

    git clone git@github.com:meiroio/me_production_environment.git

cd me_production_environment/

  6. Run Script (change the value of parameters)
    This will pull container images, check necessary requirements, create yaml template config

    ./update_production_environment.sh --https \
--domain-name=me.example.com \     #DNS should be pointing to external IP for letsecnrypt chanllege

  7. Run Docker Compose
    After Step 5, docker-compose.yml have been created

    docker-compose up

visit https://me.example.com

  8. Check for Errors
    - If you have SSL errors, make sure --domain is resolvable externally, attached the IP and rerun 5 & 6
    - Cant pull? check 3 is done already, make sure you have your IPs whitelisted to the Meiro Registry
    - Intermittent pull error? Check the network limitation of your Nat gateway or VMS
    https://cloud.google.com/nat/docs/ports-and-addresses#:~:text=Cloud%20NAT%20adjusts%20the%20minimum%20ports%20per%20VM%20instance%20if%20necessary

OpenSearch

source: https://opensearch.org/docs/2.2/opensearch/install/docker/
VM specs - 4cores, 8G ram, 160G SSD

  1. Setup Firewall access to port 22, 80, 8080, 9200, 9600, 5601

  2. ssh into the VM make use correct Specs and OS

  3. Install prerequisites.

    sudo apt install docker-compose
sudo usermod -aG docker $USER
newgrp docker

  4. Create a docker-compose file with following contents (only for sample)

    version: '3'
services:
  opensearch-node1:
    image: opensearchproject/opensearch:2.2.0
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      - "DISABLE_INSTALL_DEMO_CONFIG=true" # disables execution of install_demo_configuration.sh bundled with security plugin, which installs demo certificates and security configurations to OpenSearch
      - "DISABLE_SECURITY_PLUGIN=true" # disables security plugin entirely in OpenSearch by setting plugins.security.disabled: true in opensearch.yml
      - "discovery.type=single-node" # disables bootstrap checks that are enabled when network.host is set to a non-loopback address
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    volumes:
      - opensearch-data1:/usr/share/opensearch/data
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net

  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:2.2.0
    container_name: opensearch-dashboards
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      - 'OPENSEARCH_HOSTS=["http://opensearch-node1:9200"]'
      - "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" # disables security dashboards plugin in OpenSearch Dashboards
    networks:
      - opensearch-net

volumes:
  opensearch-data1:

networks:
  opensearch-net:

  5. Run Docker Compose

    docker-compose up

visit https://{External IP}

  6. Issues using Kube?
    Getting errors below
    max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
    This might be fix using init container and running commands

      - name: set-max-map-count
    command:
    - sh
    - -c
    - sysctl -w vm.max_map_count=262144

CockroachDB

source: https://www.cockroachlabs.com/docs/stable/deploy-cockroachdb-with-kubernetes.html#step-2-start-cockroachdb

CockroachDB(3x) - 4cores, 8G ram, 160G SSD

Run Resources via Terraform

source code: