Skip to content

Commit 95a1dd5

Browse files
sharonxsharonx
authored
Sync changes from develop to release/2.0.0 (#38)
1 parent 085e29f commit 95a1dd5

File tree

2 files changed

+299
-36
lines changed

2 files changed

+299
-36
lines changed

‎CODE_OF_CONDUCT.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Contributor Covenant Code of Conduct
2+
3+
## Our Pledge
4+
5+
In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
6+
7+
## Our Standards
8+
9+
Examples of behavior that contributes to creating a positive environment include:
10+
11+
* Using welcoming and inclusive language
12+
* Being respectful of differing viewpoints and experiences
13+
* Gracefully accepting constructive criticism
14+
* Focusing on what is best for the community
15+
* Showing empathy towards other community members
16+
17+
Examples of unacceptable behavior by participants include:
18+
19+
* The use of sexualized language or imagery and unwelcome sexual attention or advances
20+
* Trolling, insulting/derogatory comments, and personal or political attacks
21+
* Public or private harassment
22+
* Publishing others' private information, such as a physical or electronic address, without explicit permission
23+
* Other conduct which could reasonably be considered inappropriate in a professional setting
24+
25+
## Our Responsibilities
26+
27+
Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
28+
29+
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
30+
31+
## Scope
32+
33+
This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
34+
35+
## Enforcement
36+
37+
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at oss@splunk.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
38+
39+
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
40+
41+
## Attribution
42+
43+
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
44+
45+
[homepage]: http://contributor-covenant.org
46+
[version]: http://contributor-covenant.org/version/1/4/

‎README.md‎

Lines changed: 253 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,54 +1,271 @@
1-
# Splunk Log-driver plugin for Docker
21

3-
Splunk Logging Plugin allows docker containers to send their logs directly to a Splunk Enterprise service or a Splunk
4-
Cloud deployment.
2+
# What does Splunk Connect for Docker do?
3+
Splunk Connect for Docker is a plug-in that extends and expands Docker's logging capabilities so that customers can push their Docker and container logs to their Splunk on-premise or cloud deployment.
54

6-
## Getting Started
5+
Splunk Connect for Docker is a supported open source product. Customers with an active Splunk support contract receive Splunk Extension support under the Splunk Support Policy. Splunk Connect for Docker is not supported on windows. Please refer to Docker Engine managed plugin system documentation on support for Microsoft Windows and other platforms as well as Prerequisites in this document.
76

8-
You need to install Docker Engine >= 1.12.
7+
# Prerequisites
8+
Before you install Splunk Connect for Docker, make sure your system meets the following minimum prerequisites:
99

10-
Additional information about Docker plugins [can be found here.](https://docs.docker.com/engine/extend/plugins_logging/)
10+
* Docker Engine: Version 17.05 or later. If you plan to configure Splunk Connect for Docker via 'daemon.json', you must have the Docker Community Edition (Docker-ce) 18.03 equivalent or later installed.
11+
* Splunk Enterprise, Splunk Light, or Splunk Cloud version 6.6 or later. Splunk Connect for Docker plugin is not currently supported on Windows.
12+
* For customers deploying to Splunk Cloud, HEC must be enabled and a token must be generated by Splunk Support before logs can be ingested.
13+
* Configure an HEC token on Splunk Enterprise or Splunk Light (either single instance or distributed environment). Refer to the set up and use HTTP Event Collector documentation for more details.
14+
* Operating System Platform support as defined in Docker Engine managed plugin system documentation.
1115

16+
# Install and configure Splunk Connect for Docker
1217

13-
### Developing
18+
## Step 1: Get an HTTP Event Collector token
19+
You must configure the Splunk HTTP Event Collector (HEC) to send your Docker container logging data to Splunk Enterprise or Splunk Cloud. HEC uses tokens as an alternative to embedding Splunk Enterprise or Splunk Cloud credentials in your app or supporting files. For more about how the HTTP event collector works, see http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/UsetheHTTPEventCollector
1420

15-
For development, you can clone and run make
21+
1. Enable your HTTP Event collector:
22+
http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/HECWalkthrough#Enable_HEC
23+
2. Create an HEC token:
24+
http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/UsetheHTTPEventCollector
25+
http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/UseHECusingconffiles
1626

17-
```
18-
git clone git@github.com:splunk/docker-logging-plugin.git
19-
cd docker-logging-plugin
20-
make
21-
```
27+
Note the following when you generate your token:
28+
* Make sure that indexer acknowledgement is disabled for your token.
29+
* Optionally, enable the indexer acknowledgement functionality by clicking the Enable indexer management checkbox.
30+
* Do not generate your token using the default TLS cert provided by Splunk. For information about configuring Splunk to use self-signed or third-party certs, see http://docs.splunk.com/Documentation/Splunk/7.0.3/Security/AboutsecuringyourSplunkconfigurationwithSSL.
31+
* Splunk Cloud customers must file a support request in order to have a token generated.
2232

23-
### Installing
33+
## Step 2: Install the plugin
34+
There are multiple ways to install Splunk Connect for Docker, Splunk recommends installing from Docker Store (option 1) to ensure the most current and stable build.
2435

25-
To install the plugin, you can run
36+
### Install the Plugin from Docker Store
2637

27-
```
28-
docker plugin install splunk/docker-logging-driver:latest --alias splunk
29-
docker plugin ls
30-
```
38+
1. Create the plugin from the package.
3139

32-
This command will pull and enable the plugin
40+
$ docker plugin install splunk/docker-logging-plugin --alias splunk-logging-plugin
3341

34-
### Using
42+
2. Enable the plugin:
3543

36-
The plugin uses the same parameters as the [splunk logging driver](https://docs.docker.com/engine/admin/logging/splunk/).
44+
$ docker plugin enable splunk-logging-plugin:latest
3745

46+
### Install the plugin from the tar file
3847

39-
#### Splunk Enterprise Example
48+
1. Clone the repository and check out release branch
4049

41-
```
42-
$ docker run --log-driver=splunk \
43-
--log-opt splunk-url=https://your-splunkhost:8088 \
44-
--log-opt splunk-token=176FCEBF-4CF5-4EDF-91BC-703796522D20 \
45-
--log-opt splunk-capath=/path/to/cert/cacert.pem \
46-
--log-opt splunk-caname=SplunkServerDefaultCert \
47-
--log-opt tag="{{.Name}}/{{.FullID}}" \
48-
--log-opt labels=location \
49-
--log-opt env=TEST \
50-
--env "TEST=false" \
51-
--label location=west \
52-
-it ubuntu bash
50+
$ git clone https://github.com/splunk/docker-logging-plugin.git
5351

54-
```
52+
$ git checkout release/2.0.0
53+
54+
2. Create the plugin package
55+
56+
$ cd docker-logging-plugin
57+
$ make package # this creates a splunk-logging-plugin.tar.gz
58+
59+
3. unzip the package
60+
61+
$ tar -xzf splunk-logging-plugin.tar.gz
62+
63+
4. Create the plugin
64+
65+
$ docker plugin create splunk-logging-plugin:latest splunk-logging-plugin/
66+
67+
5. Verify that the plugin is installed by running the following command:
68+
69+
$ docker plugin ls
70+
71+
6. Enable the plugin
72+
73+
$ docker plugin enable splunk-logging-plugin:latest
74+
75+
## Step 3: Run containers with the plugin installed
76+
77+
Splunk Connect for Docker continually listens for logs, but your containers must also be running so that the container logs are forwarded to Splunk Connect for Docker. The following examples describe how to configure containers to run with Splunk Connect for Docker.
78+
79+
To start your containers, refer to the Docker Documentation found at:
80+
81+
https://docs.docker.com/config/containers/logging/configure/
82+
https://docs.docker.com/config/containers/logging/configure/#configure-the-delivery-mode-of-log-messages-from-container-to-log-driver
83+
84+
## Examples
85+
86+
This sample <addr>daemon.json</addr> command configures Splunk Connect for Docker for all containers on the docker engine. Splunk Software recommends that when working in a production environment, you pass your HEC token through <addr>daemon.json</addr> as opposed to the command line.
87+
88+
{
89+
"log-driver": "splunk-logging-plugin",
90+
"log-opts": {
91+
"splunk-url": "<splunk_hec_endpoint>",
92+
"splunk-token": "<splunk-hec-token>",
93+
"splunk-insecureskipverify": "true"
94+
}
95+
}
96+
97+
This sample command configures Splunk Connect for Docker for a single container.
98+
99+
$ docker run --log-driver=splunk-logging-plugin --log-opt splunk-url=<splunk_hec_endpoint> --log-opt splunk-token=<splunk-hec_token> --log-opt splunk-insecureskipverify=true -d <docker_image>
100+
101+
## Step 4: Set Configuration variables
102+
103+
Use the configuration variables to configure the behaviors and rules for Splunk Connect for Docker. For example you can confiugre your certificate security or how messages are formatted and distributed. Note the following:
104+
105+
* Configurations that pass though docker <addr>run --log-opt <opt><addr> are effective instantaneously.
106+
* You must restart the Docker engine after configuring through `<addr>`daemon.json`<addr>`
107+
108+
### How to use the variables
109+
110+
The following is an example of the logging options specified for the Splunk Enterprise instance. In this example:
111+
112+
The path to the root certificate and Common Name is specified using an HTTPS scheme to be used for verification.
113+
114+
$ docker run --log-driver=splunk-logging-plugin\
115+
--log-opt splunk-token=176FCEBF-4CF5-4EDF-91BC-703796522D20 \
116+
--log-opt splunk-url=https://splunkhost:8088 \
117+
--log-opt splunk-capath=/path/to/cert/cacert.pem \
118+
--log-opt splunk-caname=SplunkServerDefaultCert \
119+
--log-opt tag="{{.Name}}/{{.FullID}}" \
120+
--log-opt labels=location \
121+
--log-opt env=TEST \
122+
--env "TEST=false" \
123+
--label location=west \
124+
<docker_image>
125+
126+
### Required Variables
127+
128+
Variable | Description
129+
------------ | -------------
130+
splunk-token | Splunk HTTP Event Collector token.
131+
splunk-url | Path to your Splunk Enterprise, self-service Splunk Cloud instance, or Splunk Cloud managed cluster (including port and scheme used by HTTP Event Collector) in one of the following formats: https://your_splunk_instance:8088 or https://input-prd-p-XXXXXXX.cloud.splunk.com:8088 or https://http-inputs-XXXXXXXX.splunkcloud.com
132+
133+
134+
### Optional Variables
135+
136+
Variable | Description | Default
137+
------------ | ------------- | -------------
138+
splunk-source | Event source |
139+
splunk-sourcetype | Event source type |
140+
splunk-index | Event index. (Note that HEC token must be configured to accept the specified index) |
141+
splunk-capath | Path to root certificate. (Must be specified if splunk-insecureskipverify is false) |
142+
splunk-caname | Name to use for validating server certificate; by default the hostname of the splunk-url is used. |
143+
splunk-insecureskipverify| "false" means that the service certificates are validated and "true" means that server certificates are not validated. | false
144+
splunk-format | Message format. Values can be inline, json, or raw. For more infomation about formats see the Messageformats option. | inline
145+
splunk-verify-connection| Upon plug-in startup, verify that Splunk Connect for Docker can connect to Splunk HEC endpoint. False indicates that Splunk Connect for Docker will start up and continue to try to connect to HEC and will push logs to buffer until connection has been establised. Logs will roll off buffer once buffer is full. True indicates that Splunk Connect for Docker will not start up if connection to HEC cannot be established. | false
146+
splunk-gzip | Enable/disable gzip compression to send events to Splunk Enterprise or Splunk Cloud instance. | false
147+
splunk-gzip-level | Set compression level for gzip. Valid values are -1 (default), 0 (no compression), 1 (best speed) … 9 (best compression). | DefaultCompression
148+
tag | Specify tag for message, which interpret some markup. Default value is {{.ID}} (12 characters of the container ID). Refer to the log tag option documentation for customizing the log tag format. https://docs.docker.com/v17.09/engine/admin/logging/log_tags/ |
149+
labels | Comma-separated list of keys of labels, which should be included in message, if these labels are specified for container. |
150+
env | Comma-separated list of keys of environment variables to be included in message if they specified for a container. |
151+
env-regex | A regular expression to match logging-related environment variables. Used for advanced log tag options. If there is collision between the label and env keys, the value of the env takes precedence. Both options add additional fields to the attributes of a logging message. |
152+
153+
154+
### Advanced options - Environment Variables
155+
156+
Variable | Description | Default
157+
------------ | ------------- | -------------
158+
SPLUNK_LOGGING_DRIVER_POST_MESSAGES_FREQUENCY | How often plug-in posts messages when there is nothing to batch, i.e., the maximum time to wait for more messages to batch. The internal buffer used for batching is flushed either when the buffer is full (the disgnated batch size is reached) or the buffer timesout (specified by this frequency) | | 5s
159+
SPLUNK_LOGGING_DRIVER_POST_MESSAGES_BATCH_SIZE | The number of messages the plug-in should collect before sending them in one batch. | 1000
160+
SPLUNK_LOGGING_DRIVER_BUFFER_MAX | The maximum amount of messages to hold in buffer and retry when the plug-in cannot connect to remote server. | 10 * 1000
161+
SPLUNK_LOGGING_DRIVER_CHANNEL_SIZE | How many pending messages can be in the channel used to send messages to background logger worker, which batches them. | 4 * 1000
162+
SPLUNK_LOGGING_DRIVER_TEMP_MESSAGES_HOLD_DURATION | Appends logs that are chunked by docker with 16kb limit. It specifies how long the system can wait for the next message to come. | 100ms
163+
SPLUNK_LOGGING_DRIVER_TEMP_MESSAGES_BUFFER_SIZE | Appends logs that are chunked by docker with 16kb limit. It specifies the biggest message that the system can reassemble. The value provided here should be smaller than or equal to the Splunk HEC limit. 1 MB is the default HEC setting. | 1mb
164+
165+
166+
### Message formats
167+
There are three logging plug-in messaging formats set under the optional variable splunk-format:
168+
169+
* inline (this is the default format)
170+
* json
171+
* raw
172+
173+
The default format is inline, where each log message is embedded as a string and is assigned to "line" field. For example:
174+
175+
{
176+
"attrs": {
177+
"env1": "val1",
178+
"label1": "label1"
179+
},
180+
"tag": "MyImage/MyContainer",
181+
"source": "stdout",
182+
"line": "my message"
183+
}
184+
{
185+
"attrs": {
186+
"env1": "val1",
187+
"label1": "label1"
188+
},
189+
"tag": "MyImage/MyContainer",
190+
"source": "stdout",
191+
"line": "{\"foo\": \"bar\"}" //though this is a string that can be marshaled to json, it is still treated as a string
192+
}
193+
194+
When messages are JSON objects, you may want to embed them in the message sent to Splunk.
195+
196+
To format messages as json objects, set --log-opt splunk-format=json. The plug-in will try to parse every line as a JSON object and embed the json object to "line" field. If it cannot parse the message, it is sent inline. For example:
197+
198+
{
199+
"attrs": {
200+
"env1": "val1",
201+
"label1": "label1"
202+
},
203+
"tag": "MyImage/MyContainer",
204+
"source": "stdout",
205+
"line": "my message" //fall back to a string
206+
}
207+
{
208+
"attrs": {
209+
"env1": "val1",
210+
"label1": "label1"
211+
},
212+
"tag": "MyImage/MyContainer",
213+
"source": "stdout",
214+
"line": {
215+
"foo": "bar"
216+
}
217+
}
218+
219+
If --log-opt splunk-format=raw, each message together with attributes (environment variables and labels) and tags are combined in a raw string. Attributes and tags are prefixed to the message. For example:
220+
221+
#<tag> <env=value> <label=value> <logging_messaage>
222+
MyImage/MyContainer env1=val1 label1=label1 my message
223+
MyImage/MyContainer env1=val1 label1=label1 {"foo": "bar"}
224+
225+
# Troubleshooting
226+
227+
If your Splunk Connector for Docker does not behave as expected, use the debug functionality and then refer to the following tips included in output.
228+
229+
## Enable Debug Mode to find log errors
230+
231+
Plugin logs can be found as docker daemon log. To enable debug mode, export environment variable LOGGIN_LEVEL=DEBUG in docker engine environment. See the Docker documentation for information about how to enable debug mode in your docker environment: https://docs.docker.com/config/daemon/
232+
233+
## Use the debugger to check your debug the Splunk HEC connection
234+
235+
Check HEC endpoint accessibility Docker environment. If the endpoint cannot be reached, debug logs are not sent to Splunk, or the logs or will buffer and drop as they roll off the buffer.
236+
237+
Test HEC endpoint is accessible
238+
$ curl -k https://<ip_address>:8088/services/collector/health
239+
{"text":"HEC is healthy","code":200}
240+
241+
## Check your HEC configuration for clusters
242+
243+
If you are using an Indexer Cluster, the current plugin accepts a single splunk-url value. We recommend that you configure a load balancer in front of your Indexer tier. Make sure the load balancer can successfully tunnel the HEC requests to the indexer tier. If HEC is configured in an Indexer Cluster environment, all indexers should have same HEC token configured. See http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/UsetheHTTPEventCollector.
244+
245+
## Check your heavy forwarder connection
246+
247+
If you ae using a heavy forwarder to preprocess the events (e.g: funnel multiple log lines to a single event), make sure that the heavy forwarder is properly connecting to the indexers. To troubleshoot the forwarder and receiver connection, see: https://docs.splunk.com/Documentation/SplunkCloud/7.0.0/Forwarding/Receiverconnection.
248+
249+
## Check the plugin's debug log in docker
250+
251+
Stdout of a plugin is redirected to Docker logs. Such entries have a plugin=<ID> suffix.
252+
253+
To find out the plugin ID of Splunk Connect for Docker, use the command below and look for Splunk Logging Plugin entry.
254+
255+
# list all the plugins
256+
$ docker plugin ls
257+
258+
Depending on your system, location of Docker daemon logging may vary. Refer to Docker documentation for Docker daemon log location for your specific platform. Here are a few examples:
259+
260+
* Ubuntu (old using upstart ) - /var/logging/upstart/docker.logging
261+
* Ubuntu (new using systemd ) - sudo journalctl -fu docker.service
262+
* Boot2Docker - /var/logging/docker.logging
263+
* Debian GNU/Linux - /var/logging/daemon.logging
264+
* CentOS - /var/logging/daemon.logging | grep docker
265+
* CoreOS - journalctl -u docker.service
266+
* Fedora - journalctl -u docker.service
267+
* Red Hat Enterprise Linux Server - /var/logging/messages | grep docker
268+
* OpenSuSE - journalctl -u docker.service
269+
* OSX - ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/logging/d‌ocker.logging
270+
* Windows - Get-EventLog -LogName Application -Source Docker -After (Get-Date).AddMinutes(-5) | Sort-Object Time, as mentioned here.
271+

0 commit comments

Comments
 (0)