Description review
DevOps
PradeepIT Consulting Services Pvt Ltd · United States · back to the listing
HR standards
64/100
needs work
Title ↔ description
43/100
poor
Reads as
Unclear
no confident match
What this role officially is
ICT system administrator — ESCO, the EU occupation classification
ICT system administrators are responsible for the upkeep, configuration, and reliable operation of computer and network systems, servers, workstations and peripheral devices. They may acquire, install, or upgrade computer components and software; automate routine tasks; write computer programs; troubleshoot; train and supervise staff; and provide technical support. They ensure optimum system integrity, security, backup and performance.
Also known as: enterprise administrator, IT system administrator, ICT systems administrator, ICT sysadmins, IT systems administrator, ICT sysadmin
How others title the same work
Large employers
- Senior Software Engineer, Developer Infrastructure Coinbase
- Senior Software Engineer, Infrastructure - Compute Platform Coinbase
- Senior Software Engineer, Infrastructure - Core Infra Systems Coinbase
- Software Engineer - Python - Container Images Canonical Ltd.
- Software Engineer - Secure Container Images Canonical
Startups
- Ecosmic | Senior DevOps Engineer | Remote (Italy) | €50,000–€60,000 Ecosmic
- 3C Digital Solutions | Terraform Module Developer 3C Digital Solutions
- Software Engineer, Cloud Infrastructure Accord
- BCC | Platform Systems Engineers | Bethesda MD | Competitive compensation! BCC
- DevOps Engineer Cyble
What the listing never says
- 37 bullet points. Long requirement lists deter qualified candidates, who read them as hard gates. Scope clarity
- No pay range published. Candidates cannot tell whether applying is worth their time. Pay transparency
The listing, marked up
Nothing in the wording of this listing tripped a check. The scores above still judge how complete and coherent it is.
This position is within a project with one of the foundational LLM companies. The goal is to assist these foundational LLM companies in enhancing their Large Language Models.
One way we help these companies improve their models is by providing them with high-quality proprietary data. This data serves two main purposes: first, as a basis for fine-tuning their models, and second, as an evaluation set to benchmark the performance of their models or competitor models.
For example, for SFT data generation, you might have to put together or be provided a prompt which contains provided code and questions, you will then provide the model responses and write corresponding script to solve the questions. A collection of 5k-10k such samples could form the dataset for model fine-tuning.
For RLHF data generation, you might have to put together or be provided a prompt by the customer, ask the model questions, and evaluate the outputs generated by two versions of the LLM model. You'd compare these outputs and provide feedback, which is then used to fine-tune the models. Please note that this role does not require you to build LLMs or fine-tune them.
What does day-to-day look like:
• Design and develop challenging prompts based on provided source code with good coverage of DevOps and Infrastructure technologies
• Implement the verified code that can be executed to verify whether a model's response to the prompt is correct or not
• Conduct evaluations (Evals) to benchmark model performance and analyze results for continuous improvement.
• Evaluate and rank AI model responses to user queries across diverse domains, ensuring alignment with predefined criteria.
• Develop comprehensive explanations and rationales for evaluations, showcasing excellent reasoning and technical expertise.
• Lead efforts in Supervised Fine-Tuning (SFT), including creating and maintaining high-quality, task-specific datasets.
• Collaborate with researchers and annotators to execute Reinforcement Learning with Human Feedback (RLHF) and refine reward models.
• Design innovative evaluation strategies and processes to improve the model's alignment with user needs and ethical guidelines.
• Create and refine optimal responses to improve AI performance, emphasizing clarity, relevance, and technical accuracy.
• Conduct thorough peer reviews of code and documentation, providing constructive feedback and identifying areas for improvement.
• Collaborate with cross-functional teams to improve model performance and contribute to product enhancements.
• Continuously explore and integrate new tools, techniques, and methodologies to enhance AI training processes.
• Requirements:
• Technical Expertise:
• Proven 5+ yrs of experience with configuration management and infrastructure automation tools such as Ansible, Terraform, and/or similar platforms.
• Strong exposure to AWS cloud platforms with experience in designing and managing multi-cloud environments.
• Hands-on experience with container technologies (Docker) and container orchestration (Kubernetes).
• Proficiency in scripting languages (Bash, Python, etc.) for automation and tool integration.
• Familiarity with CI/CD tools (Jenkins, GitLab CI, CircleCI, etc.) and version control systems (Git).
Operational Excellence:
• Experience setting up monitoring, logging, and alerting mechanisms to ensure system health and quick incident response.
• Knowledge of networking, security best practices, and high availability design in cloud infrastructures.
• Professional Skills:
• 5+ years of overall work experience in DevOps or related roles.
• Demonstrable ability to collaborate with cross-functional teams and communicate complex technical concepts.
• Strong problem-solving skills, with a proactive approach to identifying and resolving system bottlenecks and vulnerabilities.
• Fluent in conversational and written English communication.
* Work in a fully remote environment.
• Opportunity to work on cutting-edge projects with leading AI and cloud technology companies.
• Potential for contract extension based on performance and project needs.
• Offer Details:
• Commitments Required: At least 4 hours per day and a minimum of 20 hours per week, with a 4-hour overlap with PST.
• Employment Type: Contractor position (Note: this role does not include medical/paid leave).
• Duration of Contract t: 1 month; [expected start date is next week].
• Location: Open to candidates in India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico.
• Evaluation Process (approximately 75 mins):
• Interview Rounds:
• 60-minute technical interview focusing on DevOps tools, cloud architectures, and automation strategies.
15-minute cultural fit and offer discussion session.
Originally posted on Himalayas
One way we help these companies improve their models is by providing them with high-quality proprietary data. This data serves two main purposes: first, as a basis for fine-tuning their models, and second, as an evaluation set to benchmark the performance of their models or competitor models.
For example, for SFT data generation, you might have to put together or be provided a prompt which contains provided code and questions, you will then provide the model responses and write corresponding script to solve the questions. A collection of 5k-10k such samples could form the dataset for model fine-tuning.
For RLHF data generation, you might have to put together or be provided a prompt by the customer, ask the model questions, and evaluate the outputs generated by two versions of the LLM model. You'd compare these outputs and provide feedback, which is then used to fine-tune the models. Please note that this role does not require you to build LLMs or fine-tune them.
What does day-to-day look like:
• Design and develop challenging prompts based on provided source code with good coverage of DevOps and Infrastructure technologies
• Implement the verified code that can be executed to verify whether a model's response to the prompt is correct or not
• Conduct evaluations (Evals) to benchmark model performance and analyze results for continuous improvement.
• Evaluate and rank AI model responses to user queries across diverse domains, ensuring alignment with predefined criteria.
• Develop comprehensive explanations and rationales for evaluations, showcasing excellent reasoning and technical expertise.
• Lead efforts in Supervised Fine-Tuning (SFT), including creating and maintaining high-quality, task-specific datasets.
• Collaborate with researchers and annotators to execute Reinforcement Learning with Human Feedback (RLHF) and refine reward models.
• Design innovative evaluation strategies and processes to improve the model's alignment with user needs and ethical guidelines.
• Create and refine optimal responses to improve AI performance, emphasizing clarity, relevance, and technical accuracy.
• Conduct thorough peer reviews of code and documentation, providing constructive feedback and identifying areas for improvement.
• Collaborate with cross-functional teams to improve model performance and contribute to product enhancements.
• Continuously explore and integrate new tools, techniques, and methodologies to enhance AI training processes.
• Requirements:
• Technical Expertise:
• Proven 5+ yrs of experience with configuration management and infrastructure automation tools such as Ansible, Terraform, and/or similar platforms.
• Strong exposure to AWS cloud platforms with experience in designing and managing multi-cloud environments.
• Hands-on experience with container technologies (Docker) and container orchestration (Kubernetes).
• Proficiency in scripting languages (Bash, Python, etc.) for automation and tool integration.
• Familiarity with CI/CD tools (Jenkins, GitLab CI, CircleCI, etc.) and version control systems (Git).
Operational Excellence:
• Experience setting up monitoring, logging, and alerting mechanisms to ensure system health and quick incident response.
• Knowledge of networking, security best practices, and high availability design in cloud infrastructures.
• Professional Skills:
• 5+ years of overall work experience in DevOps or related roles.
• Demonstrable ability to collaborate with cross-functional teams and communicate complex technical concepts.
• Strong problem-solving skills, with a proactive approach to identifying and resolving system bottlenecks and vulnerabilities.
• Fluent in conversational and written English communication.
* Work in a fully remote environment.
• Opportunity to work on cutting-edge projects with leading AI and cloud technology companies.
• Potential for contract extension based on performance and project needs.
• Offer Details:
• Commitments Required: At least 4 hours per day and a minimum of 20 hours per week, with a 4-hour overlap with PST.
• Employment Type: Contractor position (Note: this role does not include medical/paid leave).
• Duration of Contract t: 1 month; [expected start date is next week].
• Location: Open to candidates in India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico.
• Evaluation Process (approximately 75 mins):
• Interview Rounds:
• 60-minute technical interview focusing on DevOps tools, cloud architectures, and automation strategies.
15-minute cultural fit and offer discussion session.
Originally posted on Himalayas