ei blog

AWS SDK for JavaScript v3でよく使うDynamoDB操作の備忘録(TypeScript)

#TypeScript
#AWS SDK
#DynamoDBDocumentClient
#Jest

AWS SDK for JavaScriptは現在v3が出ていますので今回以下のパッケージを使用します

"@aws-sdk/client-dynamodb": "^3.49.0",
"@aws-sdk/lib-dynamodb": "^3.49.0",

ローカルで検証したい場合はlocalstackなどでmock環境を用意してください

localstackを利用する場合は こちら を参照してください

meta-id-idxとmeta-timestamp-idxというGSI（グローバルセカンダリーインデックス）を作成しています。 後述のqueryCommandで使用します。

const createTableInput = (tableName: string) => {
  return {
    AttributeDefinitions: [
      {
        AttributeName: "id",
        AttributeType: "S",
      },
      {
        AttributeName: "meta",
        AttributeType: "S",
      },
      {
        AttributeName: "timestamp",
        AttributeType: "N",
      },
    ],
    KeySchema: [
      {
        AttributeName: "id",
        KeyType: "HASH",
      },
      {
        AttributeName: "meta",
        KeyType: "RANGE",
      },
    ],
    GlobalSecondaryIndexes: [
      {
        IndexName: "meta-id-idx",
        KeySchema: [
          {
            AttributeName: "meta",
            KeyType: "HASH",
          },
          {
            AttributeName: "id",
            KeyType: "RANGE",
          },
        ],
        Projection: {
          ProjectionType: "ALL",
        },
        ProvisionedThroughput: {
          ReadCapacityUnits: 0,
          WriteCapacityUnits: 0,
        },
      },
      {
        IndexName: "meta-timestamp-idx",
        KeySchema: [
          {
            AttributeName: "meta",
            KeyType: "HASH",
          },
          {
            AttributeName: "timestamp",
            KeyType: "RANGE",
          },
        ],
        Projection: {
          ProjectionType: "ALL",
        },
        ProvisionedThroughput: {
          ReadCapacityUnits: 0,
          WriteCapacityUnits: 0,
        },
      }
    ],
    BillingMode: "PAY_PER_REQUEST",
    TableName: tableName,
  };
};
const tableName = "SampleTable";
await dynamoClient.send(new CreateTableCommand(createTableInput));
  • GetCommand
  • PutCommand
  • UpdateCommand
  • BatchWriteCommand
  • QueryCommand
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
import {
  DynamoDBDocumentClient,
  BatchWriteCommand,
  GetCommand, GetCommandInput, GetCommandOutput,
  UpdateCommand, UpdateCommandInput, UpdateCommandOutput,
  PutCommand, PutCommandInput, PutCommandOutput,
  QueryCommand, QueryCommandInput, QueryCommandOutput
} from "@aws-sdk/lib-dynamodb";

const defaultDdbdc = DynamoDBDocumentClient.from(new DynamoDBClient({region: "ap-northeast-1"}));

export class DynamodbTableOperation {

  static async putCommand(
    putInput: PutCommandInput,
    ddbdc = defaultDdbdc
  ): Promise<PutCommandOutput | void> {
    if (!ddbdc) {
      console.log("DDB_CLIENT_PARAM is undefined");
    }
    try {
      return await ddbdc.send(new PutCommand(putInput));
    } catch (e) {
      console.error("データ投入失敗", e);
    }
  }

  static async getCommand(
    getInput: GetCommandInput,
    ddbdc = defaultDdbdc
  ): Promise<GetCommandOutput | void> {
    if (!ddbdc) {
      console.log("DDB_CLIENT_PARAM is undefined");
    }
    try {
      return await ddbdc.send(new GetCommand(getInput));
    } catch (e) {
      console.error("データ取得失敗", e);
    }
  }

  static async updateCommand(
    updateInput: UpdateCommandInput,
    ddbdc = defaultDdbdc
  ): Promise<UpdateCommandOutput | void> {
    if (!ddbdc) {
      console.log("DDB_CLIENT_PARAM is undefined");
    }
    try {
      return await ddbdc.send(new UpdateCommand(updateInput));
    } catch (e) {
      console.error("データ更新失敗", e);
    }
  }

  static async queryCommand(
    queryInput: QueryCommandInput,
    ddbdc = defaultDdbdc
  ): Promise<QueryCommandOutput | void> {
    if (!ddbdc) {
      console.log("DDB_CLIENT_PARAM is undefined");
    }
    try {
      return await ddbdc.send(new QueryCommand(queryInput));
    } catch (e) {
      console.error("データ取得失敗", e);
    }
  }

  static async batchWriteRecords(
    records: object[],
    tableName: string,
    ddbdc = defaultDdbdc
  ): Promise<void> {
    if (!ddbdc) {
      console.log("DDB_CLIENT_PARAM is undifined");
    }
    const sliced25records = this.sliceByNumber(records, 25);
    for (const records of sliced25records) {
      const batchWriteInput = {
        RequestItems: {
          [tableName]: records.map((record) => ({
            PutRequest: {
              Item: record,
            },
          })),
        },
      };
      try {
        await ddbdc.send(new BatchWriteCommand(batchWriteInput));
      } catch (e) {
        console.error("データ投入失敗", e);
      }
    }
  }

  static sliceByNumber(array: object[], number: number) {
    const length = Math.ceil(array.length / number);
    return new Array(length).fill(null).map((_, i) =>
      array.slice(i * number, (i + 1) * number)
    );
  }
}

一つのレコードを作成するサンプルです。

const putInput = {
  TableName: "SampleTable",
  Item: {
    id: "Sample:xxxx1234",
    meta: "Sample",
    name: "sampleName",
    status: true,
    timestamp: 1639560043333,
  }
};
await DynamodbTableOperation.putCommand(putInput);

プライマリーキーを使った検索です ProjectionExpressionを設定することで項目を制御できます。 以下の例はjestのテストサンプルです。

const getInput = {
  TableName: "SampleTable",
  Key: {
    id: "Sample:xxxx1234",
    meta: "Sample"
  },
  ProjectionExpression: "#name, #timestamp",
  ExpressionAttributeNames: {
    "#name": "name",
    "#timestamp": "timestamp"}
};
const expected = {
  name: "sampleName",
  timestamp: 1639560043333
};
const response = await DynamodbTableOperation.getCommand(getInput);
expect(response.Item).toEqual(expected);

更新処理です。UpdateExpressionにて、

setを使用することで項目を更新、

removeで項目削除を行えます。

const updateInput = {
  TableName: "SampleTable",
  Key: {
    id: "Sample:xxxx1234",
    meta: "Sample"
  },
  UpdateExpression: "set #na = :na, #ts = :ts remove #st",
  ExpressionAttributeNames: {
    "#na": "name",
    "#ts": "timestamp",
    "#st": "status"
  },
  ExpressionAttributeValues: {
    ":na": "exampleName",
    ":ts": 1639560045555
  }
};
await DynamodbTableOperation.updateCommand(updateInput);

BatchWriteCommandでは25レコード(合計1MB、個別では64KB)まで一度に書き込み処理を行うことができます。

そのためbatchWriteRecordsメゾット内で25レコードずつに分割して書き込むようにしています。

レコードリスト内に同じプライマリーキーのレコードが存在する場合処理が失敗するようです。

const inputData = [
  {
    id: "Sample:xxxx0001",
    meta: "Sample",
    name: "sampleName1",
    status: true,
    timestamp: 1639560043333,
  },
  {
    id: "Sample:xxxx0002",
    meta: "Sample",
    name: "sampleName2",
    status: false,
    timestamp: 1639560043333,
  }
];
await DynamodbTableOperation.batchWriteRecords(inputData, tableName);

複数のレコードを一度に検索して取得する処理です。

　

meta-id-idxというGSIを使用してクエリをかけています。 partitionKey: meta sortKey: id

sortKeyでbegins_withを使用し、Sample:から始まるデータを取得します。

const queryInput: QueryCommandInput = {
  TableName: "SampleTable",
  IndexName: "meta-id-idx",
  KeyConditionExpression: "#meta = :meta and begins_with(#id,:prefix)",
  ExpressionAttributeNames: {
    "#meta": "meta",
    "#id": "id"
  },
  ExpressionAttributeValues: {
    ":meta": "Sample",
    ":prefix": "Sample:"
  }
};
const expected = [
  {
    id: "Sample:xxxx0001",
    meta: "Sample",
    name: "sampleName1",
    status: true,
    timestamp: 1639560043333,
  },
  {
    id: "Sample:xxxx0002",
    meta: "Sample",
    name: "sampleName2",
    status: false,
    timestamp: 1639560043333,
  }
];
const response = await DynamodbTableOperation.queryCommand(queryInput);
expect(response.Items).toEqual(expected);

　

次は、meta-timestamp-idxというGSIを使用した例です。 partitionKey: meta sortKey: timestamp

　

sortKeyでbetweenを使用し、timestampの範囲を決めて取得しています。 また、一度に1MB以上のデータは取れないためLastEvaluatedKeyを使って全てのレコードを取得するようにしています。

const queryInput: QueryCommandInput = {
  TableName: "SampleTable",
  IndexName: "meta-timestamp-idx",
  KeyConditionExpression: "#meta = :meta and #timestamp between :start and :end",
  ProjectionExpression: "#timestamp, #type, #value",
  ExpressionAttributeNames: {
    "#meta": "meta",
    "#timestamp": "timestamp",
    "#type": "type",
    "#value": "value"
  },
  ExpressionAttributeValues: {
    ":meta": `${prefix}:${tenantId}:${lineId}`,
    ":start": 0,
    ":end": 1639560053333
  }
};
let accumedResponse: {[key: string]: any}[] = [];
let response = await DynamodbTableOperation.queryCommand(queryInput);
if (!response) {
  console.log("Dynamodb query operation is failed");
  return;
}
accumedResponse = accumedResponse.concat(response.Items);
while (response && response.LastEvaluatedKey) {
  queryInput.ExclusiveStartKey = response.LastEvaluatedKey;
  response = await DynamodbTableOperation.queryCommand(queryInput);
  accumedResponse = accumedResponse.concat(response.Items);
}
const expected = [
  {
    id: "Sample:xxxx0001",
    meta: "Sample",
    name: "sampleName1",
    status: true,
    timestamp: 1639560043333,
  },
  {
    id: "Sample:xxxx0002",
    meta: "Sample",
    name: "sampleName2",
    status: false,
    timestamp: 1639560043333,
  }
];
expect(accumedResponse).toEqual(expected);

=> https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/modules/_aws_sdk_lib_dynamodb.html

← Back to home