DATA Tech Lead
Apply NowCompany: SysMind Tech
Location: North Palm Beach, FL 33408
Description:
Tech Lead Requirements:
1) Able to lead a team
A) A tech lead needs to be able to lead developers and business analysts and instruct them on what items of work are of highest priority.
B) A tech lead should be able to ensure that the work that is being done is both accurate, as well as following best practices of the EDP development structure.
2) EDP has a reference architecture that should be used for all development pipelines.
A) The Tech lead has a responsibility that any pipelines being built or designed for the projects the team is working on are following best practices of the reference architecture
B) The Tech lead should be able to use the reference architecture in order to build a proposal for how data would flow into EDP from a source system, and be consumed by the business.
i) From a development standpoint, the tech lead needs to know which services/tools are most appropriate for the project needs.
If the source data is large in records and lots of transformations are occuring, the preferred tool would be very different from a source with smaller record counts and a
more simple transformation to business use cases.
ii) From a business analyst standpoint, the tech lead should be able to guide BA's in how to document the work that is being done for future reference.
The tech lead should not only be able to explain to a BA what the project goals are, but also what the source table structure and destination table structure looks like so sample queries can be built in future business use cases more rapidly.
C) The Tech lead should be able to defend the design of their projects architecture not just for questions from technical architects, but also from source system/project owner questions.
i) As an example, the business may not be familiar with a tool being used, and the tech lead will need to explain why the tool is better integrated for our department than others the business may be familiar with.
ii) For another example, the tech lead will need to explain why one option approved by the IT department (that may be cheaper) is not ideal to use to the tech lead's managers, maybe the tool isn't as efficient, or the skill sets to develop on it are not available at that time.
3) Post production support
A) When a project goes to a support team, there is an expectation that when issues come up that the support team cannot handle, the technical lead will be able to assist.
B) Assistance can be as simple as pointing the person experiencing the issue to documentation previously developed, or can be guiding the person with the issue through proper querying methods to our database.
C) Assistance can be more complicated and involve troubleshooting pipelines that were developed a year or more before
i) depending on the issue, the tech lead should be able to determine if the issue is really intended in the original design, and to implement a "fix" a new project would need to be started
D) When a tech lead sees common issues re-occuring, they should be able to work with the ba's on the team to make documentation of the issues and point users to that documentation instead of working 1:1 with the users every time.
4) Business Interaction
A) A tech lead should be able to lead discussions with a business team as a representative of IT.
i) This means that the tech lead should be able to "translate" more technical language used in discussions with the development team into language understandable by the users/consumers of the data.
ii) This means that the tech lead will be responsible for representing our department outside of IT, and so should be able to conduct themselves in a manner that is appropriate to that audience
B) The tech lead will be required to give updates on project status to the business
i) What is on schedule
ii) What is falling behind
iii) What are the reasons we are delayed and what can the business team do to help resolve this
iv) What needs to be modified in the project's development/business analysis due to unforseen issues.
C) The tech lead is responsible for updates to the internal management on how projects are going.
i) If a project is falling behind, they are responsible for understanding what needs to be done to remedy this and making recommendations to management to do so.
5) technical knowledge
A) The tech lead will be required to be familiar with all the technologies in our reference architecture
i) For most technologies, it is ok to simply have an understanding of when the technology is applicable and when it is not to a problem
ii) Most tools are able to be learned about "on the job" but some will need to be understood before beginning with NEE
a) AWS - The tech lead should be at least competent to the cloud practitioner certification provided by AWS, though the certification is not a requirement
b) ETL - The tech lead should be familiar with ETL processes and have first-hand development experience with ETL in NEE. Specific tools are not as important as project experience.
c) API - The tech lead should be familiar with API development and how API's work. Pointing business users to our API's is the best way to ensure compliance and support.
d) SQL - The tech lead should be familiar with SQL development and be able to write queries to join/union/unload/copy/insert data without needing to reference many if any documentation
Referring to documentation for syntax with a SQL database you are not familiar with is fine, needing documentation on the types of joins and how they work is unacceptable.
90% of the data in our department is stored in a redshift database, SQL skills are a necessity to be a tech lead with EDP.
e) CICD - The tech lead should be familiar with dev/qa/prod environments and how they are used.
They should also be familiar with processes to move data/pipelines between environments efficiently
f) Consumption - Other than API's, the tech lead should be able to interpret data using various BI tools. i.e. PBi or AWS Quicksight.
B) The tech lead should be able to learn new tools as they are required for new projects
i) If there is a new project that requires ingestion from a source no existing tools can connect to, the tech lead should be able to research alternative tools that can be used.
ii) If a new tool is decided to be used over an existing one, the tech lead will need to be able to quickly learn how to use it well enough to explain it to the development team for their purposes.
6) Security
A) A tech lead is responsible for ensuring data brought into our database is secured properly according to business rules.
B) A tech lead should be able to interpret security requirements and implement them in our environments in an appropriate way.
i) i.e. When a team says that they "can see everything" the tech lead should be able to push back and understand what "everything" means to that team, then implement security to allow only access to what is necessary for their projects.
C) A tech lead will be responsible not just for business data security to the end users, but also for ensuring developers do not see data they should not.
i) i.e. when a developer is doing development on sensitive data, the tech lead
Thanks and Regards
Tarun Sharma
1) Able to lead a team
A) A tech lead needs to be able to lead developers and business analysts and instruct them on what items of work are of highest priority.
B) A tech lead should be able to ensure that the work that is being done is both accurate, as well as following best practices of the EDP development structure.
2) EDP has a reference architecture that should be used for all development pipelines.
A) The Tech lead has a responsibility that any pipelines being built or designed for the projects the team is working on are following best practices of the reference architecture
B) The Tech lead should be able to use the reference architecture in order to build a proposal for how data would flow into EDP from a source system, and be consumed by the business.
i) From a development standpoint, the tech lead needs to know which services/tools are most appropriate for the project needs.
If the source data is large in records and lots of transformations are occuring, the preferred tool would be very different from a source with smaller record counts and a
more simple transformation to business use cases.
ii) From a business analyst standpoint, the tech lead should be able to guide BA's in how to document the work that is being done for future reference.
The tech lead should not only be able to explain to a BA what the project goals are, but also what the source table structure and destination table structure looks like so sample queries can be built in future business use cases more rapidly.
C) The Tech lead should be able to defend the design of their projects architecture not just for questions from technical architects, but also from source system/project owner questions.
i) As an example, the business may not be familiar with a tool being used, and the tech lead will need to explain why the tool is better integrated for our department than others the business may be familiar with.
ii) For another example, the tech lead will need to explain why one option approved by the IT department (that may be cheaper) is not ideal to use to the tech lead's managers, maybe the tool isn't as efficient, or the skill sets to develop on it are not available at that time.
3) Post production support
A) When a project goes to a support team, there is an expectation that when issues come up that the support team cannot handle, the technical lead will be able to assist.
B) Assistance can be as simple as pointing the person experiencing the issue to documentation previously developed, or can be guiding the person with the issue through proper querying methods to our database.
C) Assistance can be more complicated and involve troubleshooting pipelines that were developed a year or more before
i) depending on the issue, the tech lead should be able to determine if the issue is really intended in the original design, and to implement a "fix" a new project would need to be started
D) When a tech lead sees common issues re-occuring, they should be able to work with the ba's on the team to make documentation of the issues and point users to that documentation instead of working 1:1 with the users every time.
4) Business Interaction
A) A tech lead should be able to lead discussions with a business team as a representative of IT.
i) This means that the tech lead should be able to "translate" more technical language used in discussions with the development team into language understandable by the users/consumers of the data.
ii) This means that the tech lead will be responsible for representing our department outside of IT, and so should be able to conduct themselves in a manner that is appropriate to that audience
B) The tech lead will be required to give updates on project status to the business
i) What is on schedule
ii) What is falling behind
iii) What are the reasons we are delayed and what can the business team do to help resolve this
iv) What needs to be modified in the project's development/business analysis due to unforseen issues.
C) The tech lead is responsible for updates to the internal management on how projects are going.
i) If a project is falling behind, they are responsible for understanding what needs to be done to remedy this and making recommendations to management to do so.
5) technical knowledge
A) The tech lead will be required to be familiar with all the technologies in our reference architecture
i) For most technologies, it is ok to simply have an understanding of when the technology is applicable and when it is not to a problem
ii) Most tools are able to be learned about "on the job" but some will need to be understood before beginning with NEE
a) AWS - The tech lead should be at least competent to the cloud practitioner certification provided by AWS, though the certification is not a requirement
b) ETL - The tech lead should be familiar with ETL processes and have first-hand development experience with ETL in NEE. Specific tools are not as important as project experience.
c) API - The tech lead should be familiar with API development and how API's work. Pointing business users to our API's is the best way to ensure compliance and support.
d) SQL - The tech lead should be familiar with SQL development and be able to write queries to join/union/unload/copy/insert data without needing to reference many if any documentation
Referring to documentation for syntax with a SQL database you are not familiar with is fine, needing documentation on the types of joins and how they work is unacceptable.
90% of the data in our department is stored in a redshift database, SQL skills are a necessity to be a tech lead with EDP.
e) CICD - The tech lead should be familiar with dev/qa/prod environments and how they are used.
They should also be familiar with processes to move data/pipelines between environments efficiently
f) Consumption - Other than API's, the tech lead should be able to interpret data using various BI tools. i.e. PBi or AWS Quicksight.
B) The tech lead should be able to learn new tools as they are required for new projects
i) If there is a new project that requires ingestion from a source no existing tools can connect to, the tech lead should be able to research alternative tools that can be used.
ii) If a new tool is decided to be used over an existing one, the tech lead will need to be able to quickly learn how to use it well enough to explain it to the development team for their purposes.
6) Security
A) A tech lead is responsible for ensuring data brought into our database is secured properly according to business rules.
B) A tech lead should be able to interpret security requirements and implement them in our environments in an appropriate way.
i) i.e. When a team says that they "can see everything" the tech lead should be able to push back and understand what "everything" means to that team, then implement security to allow only access to what is necessary for their projects.
C) A tech lead will be responsible not just for business data security to the end users, but also for ensuring developers do not see data they should not.
i) i.e. when a developer is doing development on sensitive data, the tech lead
Thanks and Regards
Tarun Sharma