1. Installation¶
You can find the video overview of this Installation section here
1.1. Terminal Installation¶
In your Terminal prompt, enter the following commands, in order:
conda create --name dbtcourse
python -m ipykernel install --user --name dbtcourse --display-name "Python (dbt Course)"
pip install --upgrade dbt==0.17.0
The above does the following:
Setting up the Conda environment “dbtcourse”
Creating a new kernel called “dbtcourse” for our Jupyter Notebooks [Optional]
installing dbt version 0.17.0. You can find out more about this verison here.
!pip install --upgrade dbt==0.17.0
Requirement already up-to-date: dbt==0.17.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (0.17.0)
Requirement already satisfied, skipping upgrade: dbt-snowflake==0.17.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt==0.17.0) (0.17.0)
Requirement already satisfied, skipping upgrade: dbt-postgres==0.17.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt==0.17.0) (0.17.0)
Requirement already satisfied, skipping upgrade: dbt-core==0.17.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt==0.17.0) (0.17.0)
Requirement already satisfied, skipping upgrade: dbt-bigquery==0.17.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt==0.17.0) (0.17.0)
Requirement already satisfied, skipping upgrade: dbt-redshift==0.17.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt==0.17.0) (0.17.0)
Requirement already satisfied, skipping upgrade: cryptography<3,>=2 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-snowflake==0.17.0->dbt==0.17.0) (2.6.1)
Requirement already satisfied, skipping upgrade: urllib3<1.26.0,>=1.20 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-snowflake==0.17.0->dbt==0.17.0) (1.24.3)
Requirement already satisfied, skipping upgrade: snowflake-connector-python==2.2.1 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-snowflake==0.17.0->dbt==0.17.0) (2.2.1)
Requirement already satisfied, skipping upgrade: azure-common<2.0.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-snowflake==0.17.0->dbt==0.17.0) (1.1.25)
Requirement already satisfied, skipping upgrade: azure-storage-blob<12.0.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-snowflake==0.17.0->dbt==0.17.0) (2.1.0)
Requirement already satisfied, skipping upgrade: psycopg2-binary~=2.8 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-postgres==0.17.0->dbt==0.17.0) (2.8.6)
Requirement already satisfied, skipping upgrade: networkx<3,>=2.3 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (2.5)
Requirement already satisfied, skipping upgrade: PyYAML>=3.11 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (5.1)
Requirement already satisfied, skipping upgrade: idna<2.9 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (2.8)
Requirement already satisfied, skipping upgrade: agate<2,>=1.6 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (1.6.1)
Requirement already satisfied, skipping upgrade: requests<2.23.0,>=2.18.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (2.21.0)
Requirement already satisfied, skipping upgrade: sqlparse<0.4,>=0.2.3 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (0.3.1)
Requirement already satisfied, skipping upgrade: json-rpc<2,>=1.12 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (1.13.0)
Requirement already satisfied, skipping upgrade: Jinja2==2.11.2 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (2.11.2)
Requirement already satisfied, skipping upgrade: colorama<0.5,>=0.3.9 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (0.4.1)
Requirement already satisfied, skipping upgrade: isodate<0.7,>=0.6 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (0.6.0)
Requirement already satisfied, skipping upgrade: hologram==0.0.7 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (0.0.7)
Requirement already satisfied, skipping upgrade: logbook<1.6,>=1.5 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (1.5.3)
Requirement already satisfied, skipping upgrade: werkzeug<0.17,>=0.15 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (0.16.1)
Requirement already satisfied, skipping upgrade: typing-extensions<3.8,>=3.7.4 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (3.7.4.3)
Requirement already satisfied, skipping upgrade: cffi<1.14,>=1.9 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (1.12.2)
Requirement already satisfied, skipping upgrade: minimal-snowplow-tracker==0.0.2 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-core==0.17.0->dbt==0.17.0) (0.0.2)
Requirement already satisfied, skipping upgrade: google-api-core<1.17.0,>=1.16.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-bigquery==0.17.0->dbt==0.17.0) (1.16.0)
Requirement already satisfied, skipping upgrade: google-cloud-core<1.4,>=1.3.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-bigquery==0.17.0->dbt==0.17.0) (1.3.0)
Requirement already satisfied, skipping upgrade: protobuf<3.12,>=3.6.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-bigquery==0.17.0->dbt==0.17.0) (3.11.3)
Requirement already satisfied, skipping upgrade: google-cloud-bigquery<1.25.0,>=1.24.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-bigquery==0.17.0->dbt==0.17.0) (1.24.0)
Requirement already satisfied, skipping upgrade: googleapis-common-protos<1.7.0,>=1.6.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-bigquery==0.17.0->dbt==0.17.0) (1.6.0)
Requirement already satisfied, skipping upgrade: six>=1.14.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-bigquery==0.17.0->dbt==0.17.0) (1.15.0)
Requirement already satisfied, skipping upgrade: boto3<1.12,>=1.4.4 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-redshift==0.17.0->dbt==0.17.0) (1.11.17)
Requirement already satisfied, skipping upgrade: botocore<1.15,>=1.5.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from dbt-redshift==0.17.0->dbt==0.17.0) (1.14.17)
Requirement already satisfied, skipping upgrade: asn1crypto>=0.21.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from cryptography<3,>=2->dbt-snowflake==0.17.0->dbt==0.17.0) (1.4.0)
Requirement already satisfied, skipping upgrade: pyjwt<2.0.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from snowflake-connector-python==2.2.1->dbt-snowflake==0.17.0->dbt==0.17.0) (1.7.1)
Requirement already satisfied, skipping upgrade: oscrypto<2.0.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from snowflake-connector-python==2.2.1->dbt-snowflake==0.17.0->dbt==0.17.0) (1.2.1)
Requirement already satisfied, skipping upgrade: certifi<2021.0.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from snowflake-connector-python==2.2.1->dbt-snowflake==0.17.0->dbt==0.17.0) (2019.3.9)
Requirement already satisfied, skipping upgrade: pytz<2021.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from snowflake-connector-python==2.2.1->dbt-snowflake==0.17.0->dbt==0.17.0) (2018.9)
Requirement already satisfied, skipping upgrade: pyOpenSSL<21.0.0,>=16.2.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from snowflake-connector-python==2.2.1->dbt-snowflake==0.17.0->dbt==0.17.0) (19.1.0)
Requirement already satisfied, skipping upgrade: pycryptodomex!=3.5.0,<4.0.0,>=3.2 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from snowflake-connector-python==2.2.1->dbt-snowflake==0.17.0->dbt==0.17.0) (3.9.8)
Requirement already satisfied, skipping upgrade: ijson<3.0.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from snowflake-connector-python==2.2.1->dbt-snowflake==0.17.0->dbt==0.17.0) (2.6.1)
Requirement already satisfied, skipping upgrade: azure-storage-common~=2.1 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from azure-storage-blob<12.0.0->dbt-snowflake==0.17.0->dbt==0.17.0) (2.1.0)
Requirement already satisfied, skipping upgrade: decorator>=4.3.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from networkx<3,>=2.3->dbt-core==0.17.0->dbt==0.17.0) (4.4.0)
Requirement already satisfied, skipping upgrade: pytimeparse>=1.1.5 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from agate<2,>=1.6->dbt-core==0.17.0->dbt==0.17.0) (1.1.8)
Requirement already satisfied, skipping upgrade: Babel>=2.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from agate<2,>=1.6->dbt-core==0.17.0->dbt==0.17.0) (2.6.0)
Requirement already satisfied, skipping upgrade: parsedatetime>=2.1 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from agate<2,>=1.6->dbt-core==0.17.0->dbt==0.17.0) (2.6)
Requirement already satisfied, skipping upgrade: python-slugify>=1.2.1 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from agate<2,>=1.6->dbt-core==0.17.0->dbt==0.17.0) (3.0.0)
Requirement already satisfied, skipping upgrade: leather>=0.3.2 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from agate<2,>=1.6->dbt-core==0.17.0->dbt==0.17.0) (0.3.3)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from requests<2.23.0,>=2.18.0->dbt-core==0.17.0->dbt==0.17.0) (3.0.4)
Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from Jinja2==2.11.2->dbt-core==0.17.0->dbt==0.17.0) (1.1.1)
Requirement already satisfied, skipping upgrade: jsonschema<3.2 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from hologram==0.0.7->dbt-core==0.17.0->dbt==0.17.0) (3.0.1)
Requirement already satisfied, skipping upgrade: python-dateutil<2.9,>=2.8 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from hologram==0.0.7->dbt-core==0.17.0->dbt==0.17.0) (2.8.0)
Requirement already satisfied, skipping upgrade: pycparser in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from cffi<1.14,>=1.9->dbt-core==0.17.0->dbt==0.17.0) (2.19)
Requirement already satisfied, skipping upgrade: setuptools>=34.0.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from google-api-core<1.17.0,>=1.16.0->dbt-bigquery==0.17.0->dbt==0.17.0) (40.6.2)
Requirement already satisfied, skipping upgrade: google-auth<2.0dev,>=0.4.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from google-api-core<1.17.0,>=1.16.0->dbt-bigquery==0.17.0->dbt==0.17.0) (1.21.1)
Requirement already satisfied, skipping upgrade: google-resumable-media<0.6dev,>=0.5.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from google-cloud-bigquery<1.25.0,>=1.24.0->dbt-bigquery==0.17.0->dbt==0.17.0) (0.5.1)
Requirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from boto3<1.12,>=1.4.4->dbt-redshift==0.17.0->dbt==0.17.0) (0.10.0)
Requirement already satisfied, skipping upgrade: s3transfer<0.4.0,>=0.3.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from boto3<1.12,>=1.4.4->dbt-redshift==0.17.0->dbt==0.17.0) (0.3.3)
Requirement already satisfied, skipping upgrade: docutils<0.16,>=0.10 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from botocore<1.15,>=1.5.0->dbt-redshift==0.17.0->dbt==0.17.0) (0.15.2)
Requirement already satisfied, skipping upgrade: text-unidecode==1.2 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from python-slugify>=1.2.1->agate<2,>=1.6->dbt-core==0.17.0->dbt==0.17.0) (1.2)
Requirement already satisfied, skipping upgrade: pyrsistent>=0.14.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from jsonschema<3.2->hologram==0.0.7->dbt-core==0.17.0->dbt==0.17.0) (0.14.11)
Requirement already satisfied, skipping upgrade: attrs>=17.4.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from jsonschema<3.2->hologram==0.0.7->dbt-core==0.17.0->dbt==0.17.0) (19.1.0)
Requirement already satisfied, skipping upgrade: pyasn1-modules>=0.2.1 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from google-auth<2.0dev,>=0.4.0->google-api-core<1.17.0,>=1.16.0->dbt-bigquery==0.17.0->dbt==0.17.0) (0.2.8)
Requirement already satisfied, skipping upgrade: cachetools<5.0,>=2.0.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from google-auth<2.0dev,>=0.4.0->google-api-core<1.17.0,>=1.16.0->dbt-bigquery==0.17.0->dbt==0.17.0) (4.1.1)
Requirement already satisfied, skipping upgrade: rsa<5,>=3.1.4; python_version >= "3.5" in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from google-auth<2.0dev,>=0.4.0->google-api-core<1.17.0,>=1.16.0->dbt-bigquery==0.17.0->dbt==0.17.0) (4.6)
Requirement already satisfied, skipping upgrade: pyasn1<0.5.0,>=0.4.6 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<2.0dev,>=0.4.0->google-api-core<1.17.0,>=1.16.0->dbt-bigquery==0.17.0->dbt==0.17.0) (0.4.8)
You are using pip version 19.0.3, however version 20.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
1.2. Verify Installation¶
Let’s see if the installation worked, below:
!dbt --version
installed version: 0.17.0
latest version: 0.18.0
Your version of dbt is out of date! You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation
Plugins:
- bigquery: 0.17.0
- snowflake: 0.17.0
- redshift: 0.17.0
- postgres: 0.17.0
!dbt --help
usage: dbt [-h] [--version] [-r RECORD_TIMING_INFO] [-d]
[--log-format {text,json,default}] [--no-write-json] [-S]
[--warn-error] [--partial-parse | --no-partial-parse]
{docs,source,init,clean,debug,deps,list,ls,snapshot,rpc,run,compile,test,seed,run-operation}
...
An ELT tool for managing your SQL transformations and data models. For more
documentation on these commands, visit: docs.getdbt.com
optional arguments:
-h, --help show this help message and exit
--version Show version information
-r RECORD_TIMING_INFO, --record-timing-info RECORD_TIMING_INFO
When this option is passed, dbt will output low-level
timing stats to the specified file. Example:
`--record-timing-info output.profile`
-d, --debug Display debug logging during dbt execution. Useful for
debugging and making bug reports.
--log-format {text,json,default}
Specify the log format, overriding the command's
default.
--no-write-json If set, skip writing the manifest and run_results.json
files to disk
-S, --strict Run schema validations at runtime. This will surface
bugs in dbt, but may incur a performance penalty.
--warn-error If dbt would normally warn, instead raise an
exception. Examples include --models that selects
nothing, deprecations, configurations with no
associated models, invalid test configurations, and
missing sources/refs in tests.
--partial-parse Allow for partial parsing by looking for and writing
to a pickle file in the target directory. This
overrides the user configuration file. WARNING: This
can result in unexpected behavior if you use
env_var()!
--no-partial-parse Disallow partial parsing. This overrides the user
configuration file.
Available sub-commands:
{docs,source,init,clean,debug,deps,list,ls,snapshot,rpc,run,compile,test,seed,run-operation}
docs Generate or serve the documentation website for your
project.
source Manage your project's sources
init Initialize a new DBT project.
clean Delete all folders in the clean-targets list (usually
the dbt_modules and target directories.)
debug Show some helpful information about dbt for debugging.
Not to be confused with the --debug option which
increases verbosity.
deps Pull the most recent version of the dependencies
listed in packages.yml
list (ls) List the resources in your project
snapshot Execute snapshots defined in your project
rpc Start a json-rpc server
run Compile SQL and execute against the current target
database.
compile Generates executable SQL from source, model, test, and
analysis files. Compiled SQL files are written to the
target/ directory.
test Runs tests on data in deployed models. Run this after
`dbt run`
seed Load data from csv files into your data warehouse.
run-operation Run the named macro with any supplied arguments.
Specify one of these sub-commands and you can find more help from there.
