SFTP
#
DescriptionCreate an API that can trigger SFTP actions.
The SFTP API can be used to pull and push files to the API server. This API makes the API server a centralized location for file transfer management
#
UIAccess the DataTrucker URL via a browser
#
Create a credentials- Go to Credentials in the sidebar > SSH Credentials
- Scroll down to Credential Management
- Fill the form to create the connection Credential name โ an arbitrary value to identify the credential object
- The hostname of the target server to SSH. Note: SSH executes from the API server
- SSH Username
- SSH password or location of the ssh private key on the API server
- Password is Privatekey, is to identify if step 6 is a password or file name ( folder hosting all your keys are defined in the server setup config files )
- SSH Port
- Click on Create
#
Create a SFTP API- Go to File API in the Sidebar > SFTP APIs.
- Fill the form to create the API
- Resource Name: an arbitrary name to identify the resource
- The Credential linked to the resources.
- Method: The type of REST Call
- SFTP Flow/location: download from a remote source to source location or upload to remote source from the source location
- Source Type: Indicates if the SFTP should move a file or a folder
- Target: location on the remote machine
- Validation Regex of Input values, input sanitization before querying
- Examples provided in the UI when you try to create a new API
#
Query the resource you created################################ query a trigger SFTP action#############################
URL: /api/v1/jobs/<resource name>TYPE: <method defined>HEADER: Authorization: "Bearer <JWT Token>"
Response: 200 OK{ jsondata....}
#############################Note: Irrespective of the method chosen, this API will not parse the body
#
As a CRD in Openshift / KubernetesFor credentials use the the API below to the management end point
---apiVersion: datatrucker.datatrucker.io/v1kind: DatatruckerFlowmetadata: name: datatruckerflow-samplespec: Resources: requests: memory: "256Mi" cpu: "250m" limits: memory: "256Mi" cpu: "500m" JobDefinitions: - resourcename: fileupload3 name: fileupload3 tenant: Admin type: File-SFTP restmethod: POST credentialname: sftpconnect1 source: download target_path: "/home/trucker/dt2/datatrucker/.gitignore_test" source_type: file source_path: "/upload/.gitignore" Keys: configmap: placeholder Scripts: configmap: placeholder Type: Job DatatruckerConfig: datatruckerconfig-sample Replicas: 1 API: name: api Image: repository: docker.io imageName: datatruckerio/datatrucker-api tagName: latest
#
API#
Create a credential via REST############################### create a SSH credential#############################
URL: /api/v1/credentials/TYPE: POSTHEADER: Authorization: "Bearer <JWT Token>"BODY (JSON): { "credentialname": "sftpconnect1", "type": "File-SFTP", "hostname": "localhost", "username": "foo", "password": "pass", "port": 22 }
Response: 201 OK
################################## create a SSH Private credential################################
URL: /api/v1/credentials/TYPE: POSTHEADER: Authorization: "Bearer <JWT Token>"BODY (JSON): { "credentialname": "sftpconnect2", "type": "File-SFTP", "hostname": "localhost", "username": "foo", "port": 22, "password": "id_rsa", // folder defined in resource.config.json "passwordisPrivateKey": true}
Response: 200 OK
#
Create the API via REST############################### create a download resource#############################
URL: /api/v1/resourcesTYPE: POSTHEADER: Authorization: "Bearer <JWT Token>"BODY (JSON): { "resourcename": "fileupload3", "type": "File-SFTP", "restmethod": "POST", "credentialname": "sftpconnect1", "source": "download", "target_path": "/home/trucker/dt2/datatrucker/.gitignore_test", "source_type": "file", "source_path": "/upload/.gitignore"}
Response: 201 OK
############################### create a upload resource#############################
URL: /api/v1/resourcesTYPE: POSTHEADER: Authorization: "Bearer <JWT Token>"BODY (JSON): { "resourcename": "fileupload", "type": "File-SFTP", "restmethod": "POST", "credentialname": "sftpconnect1", "source": "upload", "source_path": "/home/trucker/dt2/datatrucker/.gitignore", "source_type": "file", "target_path": "/upload/.gitignore"}
Response: 201 OK