GraphQL (https://graphql.earthdata.nasa.gov/api) is a modern web API built on top of Apollo (GraphQL) Server that provides access to the CMR API using GraphQL standards and syntax. It's goal is allow querying CMR data with more specificity and depth as it pertains to relational data. Given the standards based approach it lends itself well to existing and future client libraries which eases the burden on client applications and users.

Environment

Architecture 

Loading...

Components

  • GraphQL is built using Node JS and the Serverless framework
  • CMR search
  • Apollo Server

Deploying

Generally, building and deploying will be done from Bamboo. Bamboo's deployment mechanisms give the option of a script defined within the UI or a script within the repository alongside the code. GraphQL uses a script that is defined within the repository located at `bin/deploy-bamboo.sh`. The script is simply a list of commands so if Bamboo isn't an option, the commands can be ran manually via the command line. The overall process looks like this:

  1. Install necessary libraries
  2. Deploy Infrastructure
    1. Roles
  3. Deploy Application Resources
    1. Lambdas
    2. CloudWatch Events
    3. API Gateway
    4. CloudWatch Logs

Required Environment Variables For Deployments

  • AWS_ACCESS_KEY_ID AWS Access Key created in CloudTamer.
  • AWS_SECRET_ACCESS_KEY Access Token created in CloudTamer.
  • CLOUDFRONT_BUCKET_NAME NGAP dumps CloudFront to a bucket because we don't have permission to them, we use our Lambda, CloudfrontToCloudwatch, to read these logs and send them to Splunk.
  • CMR_ROOT_URL The url to the CMR endpoint to use when making queries to the API.
  • LAMBDA_TIMEOUT Value set as each Lambas timeout, a handful of Lambdas have custom timeouts for specific reasons.
  • LOG_DESTINATION_ARN AWS ARN of the log subscription provided by NGAP.
  • STAGE_NAME What label to apply to the deployment, helps separate multiple deployments within a single account. GraphQL uses the standard EED environments `sit`, `uat` and `prod`.
  • SUBNET_ID_A AWS Subnet ID provided by NGAP.
  • SUBNET_ID_B AWS Subnet ID provided by NGAP.
  • VPC_ID AWS VPC ID provided by NGAP.

Destroying

This can be done from within the AWS console by deleting the stack from CloudFormation. You may experience issues related to an S3 bucket not being empty, you can empty the bucket and continue deleting assets if you experience this. Running the commands require valid access tokens, see the Serverless Framework documentation for methods of providing these values.

Using AWS Access Tokens (Serverless Framework): https://www.serverless.com/framework/docs/providers/aws/guide/credentials#using-aws-access-keys

// Destroy Application resources and static content
serverless destroy --stage sit

// Destroy application roles
serverless destroy --stage sit --config serverless-infrastructure.yml

Configuration

GraphQL is deployed to NGAP 2.0, aka AWS using the Serverless Framework.

Backup

Code Backup

GraphQL's code base is backed up in BitBucket. BitBucket is backed up on-premise as part of EED-2 infrastructure management.

Bitbucket: https://git.earthdata.nasa.gov/projects/EDSC/repos/edsc-graphql/browse

Restoring (Deployments from Scratch)

Should GraphQL need to be completely recovered, that process takes around 30 minutes and requires a ticket that requires additional resources (NGAP).

Ticket depending on other teams:

  • AWS CloudFront endpoint for API Gateway

Once these tickets are completed:

  • Output from the ticket (API Gateway endpoint) need to be set within Bamboo.

Steps:

  1. Deploy full application
  2. Create the ticket above, the output from the deployment should be provided in the ticket (API Gateway ID)
  3. When the ticket is completed, the value will need to be saved in Bamboo.
  4. Deploy again so that the ENV variable is provided to the lambda used to process requests.

Release Cycle

GraphQL follows the SAFe process as implemented by EED. Typically, that means we plan priorities in 3 month increments and release code every 2 weeks. If needed, GraphQL can release on-demand with appropriate notice to stakeholders.

Patching and Remediation

GraphQL has a blocking step in our deployment process that audits our libraries and dependencies. Once a vulnerability is found, steps are taken to patch and update and resolve the vulnerability immediately.

In the case of a vulnerability discovered for a resource currently deployed, a ticket is filed once the issue is identified. GraphQL devops consults with the security team to prioritize the remediation of the vulnerability found. Once a ticket is created, approved, and prioritized, EDSC dev team works the issue until all vulnerabilities are resolved and deploy the updated app to all operational environments.

Diagnostics

GraphQL logs to AWS CloudWatch on a per Lambda basis; this allows you for easy access to specific Lambda logging in the event that you know which Lambda is responsible for the logs you're looking for. If a wider search needs to occur, GraphQL forwards logs to Splunk which accommodates a wider array of search abilities.

Splunk: https://logs.earthdata.nasa.gov/

Planned Maintenance

Deployments are handled via Bamboo.

Bamboo: https://ci.earthdata.nasa.gov/deploy/viewDeploymentProjectEnvironments.action?id=287375363

  • No labels