Skip to content

Commit 12dbbc4

Browse files
author
Bjorn Boe
authored
Merge pull request #86 from lacework-dev/via-scripts-update
Bug fixes, error handling, improved output and a readme.
2 parents da6ead9 + d77cc16 commit 12dbbc4

File tree

4 files changed

+196
-17
lines changed

4 files changed

+196
-17
lines changed

‎bash/README.md‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
# Handy BASH scrips for working with Lacework
2+
3+
## lw_aws_inventory.sh
4+
Script for estimating license vCPUs in an AWS environment. It leverages the AWS CLI and leverages by default the default profile that’s either configured using environment variables or configuration files in the ~/.aws folder. The script provides output in a CSV format to be imported into a spreadsheet, as well as an easy-to-read summary.
5+
6+
Note the following about the script:
7+
* It requires AWS CLI v2 to run
8+
* It does not work on Windows
9+
* It has only been verified to work on Mac and Linux based systems
10+
* It works great in a cloud shell
11+
12+
The output from running the script can look as follows:
13+
```
14+
./lw_aws_inventory.sh -p admin-account -o -r us-east-1
15+
Profile, Account ID, Regions, EC2 Instances, EC2 vCPUs, ECS Fargate Clusters, ECS Fargate Running Containers/Tasks, ECS Fargate CPU Units, ECS Fargate License vCPUs, Lambda Functions, MB Lambda Memory, Lambda License vCPUs, Total vCPUSs
16+
sandbox-1, 123456789012, us-east-1, 2, 2, 0, 0, 0, 0, 0, 0, 0, 2
17+
sandbox-2, 234567890123, us-east-1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
18+
logging, 345678901234, us-east-1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
19+
######################################################################
20+
Lacework inventory collection complete.
21+
22+
Organizations Analyzed: 1
23+
Accounts Analyzed: 3
24+
25+
EC2 Information
26+
====================
27+
EC2 Instances: 2
28+
EC2 vCPUs: 2
29+
30+
Fargate Information
31+
====================
32+
ECS Clusters: 0
33+
ECS Fargate Running Tasks: 0
34+
ECS Fargate Container CPU Units: 0
35+
ECS Fargate vCPUs: 0
36+
37+
Lambda Information
38+
====================
39+
Lambda Functions: 0
40+
MB Lambda Memory: 0
41+
Lambda License vCPUs: 0
42+
43+
License Summary
44+
====================
45+
EC2 vCPUs: 2
46+
+ ECS Fargate vCPUs: 0
47+
+ Lambda License vCPUs: 0
48+
----------------------------
49+
= Total vCPUs: 2
50+
```
51+
The following options can be used to modify how the script is run:
52+
### Specify one or more account profiles to scan using -p parameter
53+
```
54+
./lw_aws_inventory.sh -p default,lw-customerdemo
55+
```
56+
### Specify what regions to scan, to speed up scanning or avoid restricted regions
57+
```
58+
./lw_aws_inventory.sh -r us-east-1,us-east-2
59+
```
60+
### Scan all accounts in an AWS Organization
61+
```
62+
./lw_aws_inventory.sh -o
63+
```
64+
This will leverage the OrganizationAccountAccessRole to scan all accounts in an organization.
65+
66+
## lw_gcp_inventory.sh
67+
Script for estimating license vCPUs in a GCP environment, based on folder, project or organization level.
68+
69+
Note the following about the script:
70+
* It does not work on Windows
71+
* It has only been verified to work on Mac and Linux based systems
72+
* It works great in a cloud shell
73+
74+
```
75+
$ ./lw_gcp_inventory.sh -help
76+
Usage: ./lw_gcp_inventory.sh [-f folder] [-o organization] [-p project]
77+
Any single scope can have multiple values comma delimited, but multiple scopes cannot be defined.
78+
```
79+
80+
By default, the script will scan any project that the user has access to:
81+
```
82+
$ ./lw_gcp_inventory.sh
83+
"Project", "VM Count", "vCPUs"
84+
"projects/project-one", 2, 8
85+
"projects/project-two", 3, 12
86+
##########################################
87+
Lacework inventory collection complete.
88+
89+
License Summary:
90+
================================================
91+
Number of VMs, including standard GKE: 5
92+
vCPUs: 20
93+
```
94+
95+
The scope of the scan can be further refined using the -f, -o or -p parameters:
96+
```
97+
$ ./lw_gcp_inventory.sh -p project-one,project-two
98+
"Project", "VM Count", "vCPUs"
99+
"projects/project-one", 2, 8
100+
"projects/project-two", 3, 12
101+
##########################################
102+
Lacework inventory collection complete.
103+
104+
License Summary:
105+
================================================
106+
Number of VMs, including standard GKE: 5
107+
vCPUs: 20
108+
```
109+
110+
## lw_azure_inventory.sh
111+
Script for estimating license vCPUs in an Azure environment, based on folder, project or organization level.
112+
113+
Note the following about the script:
114+
* It does not work on Windows
115+
* It has only been verified to work on Mac and Linux based systems
116+
* It works great in a cloud shell
117+
118+
```
119+
./lw_azure_inventory.sh -help
120+
Usage: ./lw_azure_inventory.sh [-m management_group] [-s subscription]
121+
Any single scope can have multiple values comma delimited, but multiple scopes cannot be defined.
122+
```
123+
124+
By default, the script will scan any subscriptions the user has configured access to:
125+
```
126+
$ ./lw_azure_inventory.sh -m b448f327-c977-4cb8-9c27-09cfaa781bb9
127+
resource-graph extension already present...
128+
Building Azure VM SKU to vCPU map...
129+
Map built successfully.
130+
Load subscriptions
131+
Load VMs
132+
Load VMSS
133+
"Subscription ID", "Subscription Name", "VM Instances", "VM vCPUs", "VM Scale Sets", "VM Scale Set Instances", "VM Scale Set vCPUs", "Total Subscription vCPUs"
134+
"1215ba55...", "Subscription Number One", 2, 4, 0, 0, 0, 4
135+
"72165fcf...", "Subscription Number Two", 1, 2, 0, 0, 0, 2
136+
##########################################
137+
Lacework inventory collection complete.
138+
139+
VM Summary:
140+
===============================
141+
VM Instances: 3
142+
VM vCPUS: 6
143+
144+
VM Scale Set Summary:
145+
===============================
146+
VM Scale Sets: 0
147+
VM Scale Set Instances: 0
148+
VM Scale Set vCPUs: 0
149+
150+
License Summary
151+
===============================
152+
VM vCPUS: 6
153+
+ VM Scale Set vCPUs: 0
154+
-------------------------------
155+
Total vCPUs: 6
156+
```
157+
158+
The scope can further be refined by specifying management groups or subscriptions.
159+
### Specify subscriptions to scan
160+
```
161+
$ ./lw_azure_inventory.sh -s 1215ba55,72165fcf
162+
```
163+
### Specify management group to scan
164+
```
165+
$ ./lw_azure_inventory.sh -m mymanagementgroup,myothermanagementgroup
166+
```

‎bash/lw_aws_inventory.sh‎

Lines changed: 26 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#!/bin/bash
22
# Script to fetch AWS inventory for Lacework sizing.
3-
# Requirements: awscli, jq
3+
# Requirements: awscli v2, jq
44

55
# You can specify a profile with the -p flag and to scan an AWS organization using the -o flag
66
# Note:
@@ -46,6 +46,15 @@ while getopts ":p:or:" opt; do
4646
done
4747
shift $((OPTIND -1))
4848

49+
#Check AWS CLI pre-requisites
50+
AWS_CLI_VERSION=$(aws --version 2>&1 | cut -d " " -f1 | cut -d "/" -f2)
51+
if [[ $AWS_CLI_VERSION = 1* ]]
52+
then
53+
echo The script requires AWS CLI v2 to run. The current version installed is version $AWS_CLI_VERSION.
54+
echo See https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html for instructions on how to upgrade.
55+
exit
56+
fi
57+
4958
# Set the initial counts to zero.
5059
ACCOUNTS=0
5160
ORGANIZATIONS=0
@@ -87,10 +96,12 @@ function getECSFargateRunningTasks {
8796
RUNNING_FARGATE_TASKS=0
8897
for c in $ecsfargateclusters; do
8998
allclustertasks=$(aws $profile_string ecs list-tasks --region $r --output json --cluster $c --no-cli-pager | jq -r '.taskArns | join(" ")')
90-
if [ -n "${allclustertasks}" ]; then
91-
fargaterunningtasks=$(aws $profile_string ecs describe-tasks --region $r --output json --tasks $allclustertasks --cluster $c --no-cli-pager | jq '[.tasks[] | select(.launchType=="FARGATE") | select(.lastStatus=="RUNNING")] | length')
92-
RUNNING_FARGATE_TASKS=$(($RUNNING_FARGATE_TASKS + $fargaterunningtasks))
93-
fi
99+
while read -r batch; do
100+
if [ -n "${batch}" ]; then
101+
fargaterunningtasks=$(aws $profile_string ecs describe-tasks --region $r --output json --tasks $batch --cluster $c --no-cli-pager | jq '[.tasks[] | select(.launchType=="FARGATE") | select(.lastStatus=="RUNNING")] | length')
102+
RUNNING_FARGATE_TASKS=$(($RUNNING_FARGATE_TASKS + $fargaterunningtasks))
103+
fi
104+
done < <(echo $allclustertasks | xargs -n 90)
94105
done
95106

96107
echo "${RUNNING_FARGATE_TASKS}"
@@ -100,13 +111,15 @@ function getECSFargateRunningCPUs {
100111
RUNNING_FARGATE_CPUS=0
101112
for c in $ecsfargateclusters; do
102113
allclustertasks=$(aws $profile_string ecs list-tasks --region $r --output json --cluster $c --no-cli-pager | jq -r '.taskArns | join(" ")')
103-
if [ -n "${allclustertasks}" ]; then
104-
cpucounts=$(aws $profile_string ecs describe-tasks --region $r --output json --tasks $allclustertasks --cluster $c --no-cli-pager | jq '[.tasks[] | select(.launchType=="FARGATE") | select(.lastStatus=="RUNNING")] | .[].cpu | tonumber')
105-
106-
for cpucount in $cpucounts; do
107-
RUNNING_FARGATE_CPUS=$(($RUNNING_FARGATE_CPUS + $cpucount))
108-
done
109-
fi
114+
while read -r batch; do
115+
if [ -n "${batch}" ]; then
116+
cpucounts=$(aws $profile_string ecs describe-tasks --region $r --output json --tasks $batch --cluster $c --no-cli-pager | jq '[.tasks[] | select(.launchType=="FARGATE") | select(.lastStatus=="RUNNING")] | .[].cpu | tonumber')
117+
118+
for cpucount in $cpucounts; do
119+
RUNNING_FARGATE_CPUS=$(($RUNNING_FARGATE_CPUS + $cpucount))
120+
done
121+
fi
122+
done < <(echo $allclustertasks | xargs -n 90)
110123
done
111124

112125
echo "${RUNNING_FARGATE_CPUS}"
@@ -230,7 +243,7 @@ function doAnalyzeOrganization {
230243
for account in $(echo $accounts | jq -r '.Id')
231244
do
232245
ACCOUNTS=$(($ACCOUNTS + 1))
233-
local account_name=$(echo $accounts | jq -c | grep $account | jq -r '.Name')
246+
local account_name=$(echo $accounts | jq -r --arg account "$account" 'select(.Id==$account) | .Name')
234247
local account_credentials=$(aws $org_profile_string sts assume-role --role-session-name LW-INVENTORY --role-arn arn:aws:iam::$account:role/OrganizationAccountAccessRole)
235248

236249
export AWS_ACCESS_KEY_ID=$(echo $account_credentials | jq -r '.Credentials.AccessKeyId')

‎bash/lw_azure_inventory.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,11 +17,11 @@ while getopts ":m:s:" opt; do
1717
MANAGEMENT_GROUP=$OPTARG
1818
;;
1919
\? )
20-
echo "Usage: ./lw_azure_inventory.sh [-m management_group] [-s subscription] \nAny single scope can have multiple values comma delimited, but multiple scopes cannot be defined." 1>&2
20+
printf "Usage: ./lw_azure_inventory.sh [-m management_group] [-s subscription] \nAny single scope can have multiple values comma delimited, but multiple scopes cannot be defined.\n" 1>&2
2121
exit 1
2222
;;
2323
: )
24-
echo "Usage: ./lw_azure_inventory.sh [-m management_group] [-s subscription] \nAny single scope can have multiple values comma delimited, but multiple scopes cannot be defined." 1>&2
24+
printf "Usage: ./lw_azure_inventory.sh [-m management_group] [-s subscription] \nAny single scope can have multiple values comma delimited, but multiple scopes cannot be defined.\n" 1>&2
2525
exit 1
2626
;;
2727
esac

‎bash/lw_gcp_inventory.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,11 @@ while getopts ":f:o:p:" opt; do
2020
PROJECTS=$OPTARG
2121
;;
2222
\? )
23-
echo "Usage: ./lw_gcp_inventory.sh [-f folder] [-o organization] [-p project] \nAny single scope can have multiple values comma delimited, but multiple scopes cannot be defined." 1>&2
23+
printf "Usage: ./lw_gcp_inventory.sh [-f folder] [-o organization] [-p project] \nAny single scope can have multiple values comma delimited, but multiple scopes cannot be defined.\n" 1>&2
2424
exit 1
2525
;;
2626
: )
27-
echo "Usage: ./lw_gcp_inventory.sh [-f folder] [-o organization] [-p project] \nAny single scope can have multiple values comma delimited, but multiple scopes cannot be defined." 1>&2
27+
printf "Usage: ./lw_gcp_inventory.sh [-f folder] [-o organization] [-p project] \nAny single scope can have multiple values comma delimited, but multiple scopes cannot be defined.\n" 1>&2
2828
exit 1
2929
;;
3030
esac

0 commit comments

Comments
 (0)