This documentation describes an old release, version 3.5.0. Documentation for the latest release, 3.6.1, can be found here.

builtin.nosql.nosql.iov2.mongodb.MongoDbInput

MongoDbInput

Get records from MongoDB database.

MongoDbInput can be specified as an inputter in a job configuration via the use parameter:

use: nosql.iov2#MongoDbInput

Examples:

Simple find:

{
    "input": {
        "use": "nosql.iov2#MongoDbInput",
        "database": "analytics",
        "collection": "users",
        "host": "mongodb://USER:PASSWORD@host",
        "find": {
            "name": "USERNAME"
        },
        debug: False
    },
    "output": {}
}

To find by ObjectID:

{
    "input": {
        "use": "nosql.iov2#MongoDbInput",
        "database": "analytics",
        "collection": "users",
        "host": "mongodb://USER:PASSWORD@host",
        "find": {
            "_id": {
                "$oid": "611277960f56ac5d0b9ada68"
            }
        },
        debug: False
    }
}

To find by ISODate:

{
    "input": {
        "use": "nosql.iov2#MongoDbInput",
        "database": "analytics",
        "collection": "users",
        "host": "mongodb://USER:PASSWORD@host",
        "find": {
            "my_date_column": {
                "$date": "1985-04-12T23:20:50.52Z"
            }
        }
    }
}

Aggregate:

{
    "input": {
        "use": "nosql.iov2#MongoDbInput",
        "database": "analytics",
        "collection": "users",
        "host": "mongodb://USER:PASSWORD@host",
        "aggregate": [
            {"$match": {
                 "updated_at": {
                    "$gte": {"$date": "2013-10-01T00:00:00.000Z"}},
            {"$match": { "status": "A" } }
        ]
    },
    "output": {}
}

Upsert:

{
    "input": {
        "use": "nosql.iov2#MongoDbInput",
        "database": "analytics",
        "collection": "users",
        "host": "mongodb://USER:PASSWORD@host",
        "find": {
            "age": {"$gte": 18},
        }
        "upsert": {
            "last_updated_isodate": {
                "$gte": {
                    "$date": "2021-01-25T22:43:01.884000+00:00"}}}
    },
    "output": {}
}

Limit:

{
    "input": {
        "use": "nosql.iov2#MongoDbInput",
        "database": "analytics",
        "collection": "pictures",
        "host": "mongodb://USER:PASSWORD@host",
        "find": {
            "color": "red",
        }
        "limit": 3
    },
    "output": {}
}

type

object

properties

  • host

Host

MongoDB database host URL to connect to

type

string

maxLength

65536

minLength

1

format

uri

  • database

Database

MongoDB database name to connect to

type

string

  • credentials

Credentials

MongoDB database credentials to connect to the db

default

anyOf

type

object

type

string

  • collection

Collection

MongoDB collection name records will be searched in

type

string

  • find

Find

A JSON string to search records.

type

object

default

  • aggregate

Aggregate

A JSON string to aggregate records.

default

anyOf

type

object

type

array

items

  • upsert

Upsert

Upsert object to perform upsert on MongoDB. This can be a:

  • string (e.g. my_date_column)

  • Extended JSON dict :

    1. {“$gte”: {“$date”: “%(timestamp)s”}}

    2. {“$gt”: {“$date”: “2020-04-30T00:00:00.000Z”}}

Please note that if you use the exact value for the $date it should be in ISO-8601 Date/Time Format. The timestamp argument would be auto replaced with the datetime the store was modified last time.

anyOf

type

string

type

object

  • limit

Limit

An integer that determines how many records to fetch from MongoDB

type

integer

exclusiveMinimum

0

default

0

  • debug

Debug

Determines if to log JSON queries sent to the MongoDB

type

boolean

default

True

  • timestamp

Timestamp

A timestamp when the store was last modified.This is populated automatically

type

string

format

date-time