AWS CloudTrail でイベント履歴を手動でダウンロードする CSV と、Python でダウンロードする CSV が Resource を除いて、ほぼ同一になることを目指してコードを書いてみました。Error code 列のみ、どんな値が入るのか不明なので空欄としています。前日から当日のイベント履歴を全リージョン分ダウンロードしてみたところ、イベント履歴の量に比例しますが、自分の環境では 4 分程度となりました。

手動 CSV ダウンロードする場合の設定状態

全リージョンで指定日数分の CSV ダウンロードする Python コード

import boto3
import csv
import json
from datetime import datetime, timedelta, timezone

# ログ取得する日数（最大90日）
latest_days = 1

# AWSクライアントの作成
session = boto3.Session()

# 利用可能なすべてのリージョンを取得
ec2 = session.client('ec2')
regions = sorted([region['RegionName'] for region in ec2.describe_regions()['Regions']])

# 90日前の日付（時間は00:00に設定）
start_date = (datetime.now(timezone.utc) - timedelta(days=latest_days)).replace(hour=0, minute=0, second=0, microsecond=0)

# 出力するフィールドの定義
output_fields = [
    "User name", "AWS access key", "Event time", "Event source", "Event name",
    "AWS region", "Source IP address", "User agent", "Error code", "Resources",
    "Request ID", "Event ID", "Read-only", "Event type", "Recipient Account Id",
    "Event category"
]

for region in regions:
    print(f"region: {region}")

    # CloudTrailクライアントをリージョンごとに作成
    cloudtrail = session.client('cloudtrail', region_name=region)

    # 1日ずつ処理するためのループ
    for day in range(latest_days+1):
        # 各日の開始時間と終了時間を設定
        day_start = start_date + timedelta(days=day)
        day_end = day_start + timedelta(days=1)

        # CSVファイル名を作成
        csv_filename = f'output/cloudtrail_events_{day_start.strftime("%Y-%m-%d")}_{region}.csv'

        print(f"{csv_filename} {day_start} to {day_end}")

        with open(csv_filename, mode='w', newline='') as file:
            writer = csv.DictWriter(file, fieldnames=output_fields)
            writer.writeheader()  # ヘッダーを書き込む
            
            next_token = None

            while True:
                # CloudTrailイベントの取得（ページネーション対応）
                if next_token:
                    response = cloudtrail.lookup_events(
                        StartTime=day_start,
                        EndTime=day_end,
                        MaxResults=50,
                        NextToken=next_token  # 次ページがある場合に使用
                    )
                else:
                    response = cloudtrail.lookup_events(
                        StartTime=day_start,
                        EndTime=day_end,
                        MaxResults=50
                    )

                # 各イベントを書き込むためにフォーマットする
                for event in response['Events']:
                    # print(event)
                    utc_time = event['EventTime'].astimezone(timezone.utc)
                    cloudtrail_event = json.loads(event['CloudTrailEvent'])
                    formatted_event = {
                        "User name": event.get('Username', ''),
                        "AWS access key": event.get('AccessKeyId', ''),
                        "Event time": utc_time.strftime('%Y-%m-%dT%H:%M:%SZ'),
                        "Event source": event['EventSource'],
                        "Event name": event['EventName'],
                        "AWS region": cloudtrail_event.get('awsRegion', ''),
                        "Source IP address": cloudtrail_event.get('sourceIPAddress', ''),
                        "User agent": cloudtrail_event.get('userAgent', ''),
                        "Error code": '',
                        "Resources": event['Resources'],
                        "Request ID": cloudtrail_event.get('requestID', ''),
                        "Event ID": event['EventId'],
                        "Read-only": event['ReadOnly'],
                        "Event type": cloudtrail_event.get('eventType', ''),
                        "Recipient Account Id": cloudtrail_event.get('recipientAccountId', ''),
                        "Event category": cloudtrail_event.get('eventCategory', ''),
                    }
                    writer.writerow(formatted_event)
                
                # 次ページがあるか確認
                next_token = response.get('NextToken')
                if not next_token:
                    break

実行状況

% time python cloudtrail-loookup-events.py
region: ap-northeast-1
output/cloudtrail_events_2024-11-08_ap-northeast-1.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_ap-northeast-1.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: ap-northeast-2
output/cloudtrail_events_2024-11-08_ap-northeast-2.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_ap-northeast-2.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: ap-northeast-3
output/cloudtrail_events_2024-11-08_ap-northeast-3.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_ap-northeast-3.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: ap-south-1
output/cloudtrail_events_2024-11-08_ap-south-1.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_ap-south-1.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: ap-southeast-1
output/cloudtrail_events_2024-11-08_ap-southeast-1.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_ap-southeast-1.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: ap-southeast-2
output/cloudtrail_events_2024-11-08_ap-southeast-2.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_ap-southeast-2.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: ca-central-1
output/cloudtrail_events_2024-11-08_ca-central-1.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_ca-central-1.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: eu-central-1
output/cloudtrail_events_2024-11-08_eu-central-1.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_eu-central-1.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: eu-north-1
output/cloudtrail_events_2024-11-08_eu-north-1.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_eu-north-1.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: eu-west-1
output/cloudtrail_events_2024-11-08_eu-west-1.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_eu-west-1.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: eu-west-2
output/cloudtrail_events_2024-11-08_eu-west-2.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_eu-west-2.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: eu-west-3
output/cloudtrail_events_2024-11-08_eu-west-3.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_eu-west-3.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: sa-east-1
output/cloudtrail_events_2024-11-08_sa-east-1.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_sa-east-1.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: us-east-1
output/cloudtrail_events_2024-11-08_us-east-1.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_us-east-1.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: us-east-2
output/cloudtrail_events_2024-11-08_us-east-2.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_us-east-2.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: us-west-1
output/cloudtrail_events_2024-11-08_us-west-1.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_us-west-1.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
region: us-west-2
output/cloudtrail_events_2024-11-08_us-west-2.csv 2024-11-08 00:00:00+00:00 to 2024-11-09 00:00:00+00:00
output/cloudtrail_events_2024-11-09_us-west-2.csv 2024-11-09 00:00:00+00:00 to 2024-11-10 00:00:00+00:00
python cloudtrail-loookup-events.py  6.12s user 0.51s system 2% cpu 3:56.62 total

手動ダウンロードと自動ダウンロードで CSV の中身を比較

参考

https://docs.aws.amazon.com/ja_jp/awscloudtrail/latest/userguide/view-cloudtrail-events-cli.html

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudtrail/client/lookup_events.html

https://docs.aws.amazon.com/ja_jp/awscloudtrail/latest/APIReference/API_LookupEvents.html

タグ: AWS, CloudTrail, python